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Abstract 


This  report  investigates  the  current  approaches  to  Concept  of  Operations  (CONOPS) 
development  in  use  in  various  DoD  and  commercial  organizations  with  the  goal  of 
understanding  why  CONOPS  creation  is  such  a  lengthy  process,  and  how  the  process 
can  be  made  more  agile.  A  number  of  CONOPS  are  cataloged  and  analyzed  to 
understand  which  parts  of  the  current  standards  are  used  by  the  creators  of  a  CONOPS. 
Traditional  CONOPS  creation  processes  are  discussed  based  on  literature  and  face-to- 
face  interviews  with  those  involved  with  creating  CONOPS  in  both  traditional  and  non- 
traditional  domains.  Based  on  these  findings,  an  agile  CONOPS  process  that  emphasizes 
stakeholder  involvement  and  expedites  shared  mental  models  development  is  put  forth. 
Additionally,  current  and  emerging  technologies  that  might  be  applicable  to  creating  a 
graphical  CONOPS  are  discussed.  Finally,  recommendations  for  future  research  to 
develop  a  toolbox  for  creating  graphical  CONOPS  are  presented. 
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1  Summary 


As  the  bioterrorism  program  lead  for  a  federal  agency,  you  have  been  asked  to  draft  a 
concept  of  operations  (CONOPS)  for  an  emergency  response  grid  (ERG)  that  allows  fast 
detection  of  and  response  to  biological  attacks  on  urban  area.  You  log  into  the  web- 
based  agile  CONOPS  system  (ACS).  After  logging  in,  you  are  prompted  to  (l)  prepare  an 
objective  statement  describing  the  purpose  of  the  system,  responsibilities  of  the  team, 
and  scope  of  the  project  and  (2)  identity  potential  team  members  and  schedule  the  first 
meeting.  The  process  has  started... 

Day  1-5:  At  their  first  session,  the  team,  using  ACS,  identifies  potential  stakeholders 
within  the  DHS,  the  CDC,  state  emergency  response  agencies,  city  governments, 
individuals  with  expertise  relevant  to  ERGs,  and  the  system  architect(s);  and  determines 
the  relative  participation  levels  appropriate  for  each.  As  the  stakeholders  are  contacted 
the  team  uses  ACS  to  define  an  initial  concrete  problem  statement  in  a  graphical 
representation. 

Days  6-13:  All  stakeholders  who  have  responded  and  have  been  approved  to  participate 
are  given  access  to  ACS.  They  find  the  problem  statement  and  a  link  to  a  protected  page 
where  they  map  their  desired  conceptual  characteristics  for  the  ERG  system  graphically. 

Days  14-20:  The  team  consolidates  all  stakeholder  input  and  calls  a  joint  meeting  with 
the  stakeholders,  through  a  combination  of  tele-  and  video-conferencing  with  shared 
access  to  the  ACS  graphical  interface,  to  finalize  a  conceptual  view  of  the  desired  ERG. 
Feedback  is  sought  immediately  from  stakeholders  unable  to  attend  the  meeting.  All 
feedback  is  incorporated  as  comments  on  the  graphical  representation.  While  there  are 
a  few  challenges  (e.g.,  stakeholders’  untimely  responses  and  dynamics  among  some 
stakeholders)  you  realize  this  must  be  handled  in  an  agile  manner  due  to  the  evolving 
nature  of  the  problem,  and  those  challenges  will  be  overcome  using  an  agile  approach. 

In  the  meantime,  the  team  is  gathering  technical  data  necessary  for  the  next  step. 

Days  21-27:  Using  ACS  tools  and  the  agreed  upon  conceptual  view,  the  team  produces 
an  initial  graphical  representation  so  that  the  stakeholders  can  visualize  the  concept  of 
operations.  They  then  provide  their  inputs,  and  the  CONOPS  evolves  according  to  the 
stakeholder  needs  and  understanding.  Through  a  series  of  joint  sessions,  the 
stakeholders  negotiate  various  capabilities,  evaluate  tradeoffs,  and  assess  risks  using 
tools  embedded  in  the  ACS  such  as  the  multi-attribute  trade-off  analysis  tool.  Within  a 
week,  stakeholders  converge  on  a  set  of  prioritized  capabilities  for  the  ERG  system 
including  detecting  small  traces  of  the  pathogens,  measuring  wind  velocity,  assessing 
chemical  composition,  providing  video  imagery,  communicating  incident  information  in 
a  trustworthy  and  secure  information  network,  GIS  mapping  and  modeling  of 
atmospheric  conditions.  Some  stakeholders  insist  on  including  a  training  simulation 
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module,  but  given  the  additional  cost  this  capability  is  deferred  to  later  upgrades  of  the 
system. 

Days  28-41:  Based  on  the  agreed  upon  capabilities,  the  team  prepares  a  list  of  potential 
system  components  and  a  draft  management  plan.  In  three  sessions  of  stakeholder 
negotiations,  the  system  component  list  is  refined  and  the  system  architecture  is 
defined.  The  management  plan  is  revised  based  on  the  availability  of  organizational 
resources  involved  in  ERG  deployment/management  (as  reported  by  stakeholders).  The 
team  finalizes  all  documentation. 

Days  42-55:  Stakeholders  are  provided  with  links  to  the  system  architecture, 
management  plan  and  all  artifacts  of  the  CONOPS  process  within  ACS  and  asked  to 
solicit  feedback  within  their  respective  organizations.  In  two  joint  sessions  during  this 
time  period,  stakeholders  discuss  their  respective  organizational  inputs,  resolve  any 
conflicts  among  inputs,  and  agree  on  a  final  system  architecture  and  management  plan. 

Day  56:  The  ACS  has  enabled  you  to  automatically  generate  a  standard  CONOPS  report 
(in  either  the  IEEE  or  DoD  format)  and  you  can  deliver  it  to  your  boss  for  final  approval. 
Your  boss  is  pleased  to  see  that  the  final  document  is  the  result  of  input  from  over  20 
stakeholders  representing  relevant  constituents  and  commends  you  on  completing  the 
CONOPS  in  less  than  two  months.  The  standard  CONOPS  document,  models  and 
scenario  are  then  sent  to  the  business  units  for  final  review  prior  to  acquisition 
processes  getting  started. 

This  report  describes  the  research  behind  the  fictitious,  but  possible,  ACS  system 
discussed  above.  It  is  based  on  a  survey  of  23  openly  available  CONOPS  across  multiple 
industries  and  disciplines  to  understand  what  is  important  in  a  concept  of  operations, 
and  what  is  not  used.  The  report  describes  a  process  for  arriving  at  a  shared  vision 
across  multiple  stakeholders.  Currently  there  is  no  off-the-shelf  tool  to  perform  this 
task,  but  research  is  presented  to  assess  the  state  of  the  technology  for  automating  the 
process,  and  recommending  a  way  ahead  for  further  research,  and  prototyping  of  some 
aspects  of  an  ACS  like  solution. 


Contract  Number:  H98230-08-D-0171  DO  001,10  002  RT3  CONOPS 

Report  No.  SERC- 2009- TR- 003 
October  30,  2009 


UNCLASSIFIED 

9 


UNCLASSIFIED 


2  Introduction 


This  report  is  the  result  of  a  three  month  research  task  for  the  Systems  Engineering 
Research  Center1  (SERC)  to  investigate  what  would  be  involved  in  producing  a  graphical 
CONOPS  (Concept  of  Operations)  development  environment  for  agile  systems 
engineering. 


2.1  Problem  Statement 

CONOPS,  if  created  on  programs,  are  often  generated  in  the  form  of  lengthy  documents 
supported  by  static  images  often  developed  in  PowerPoint.  This  is  a  labor-intensive, 
time-consuming  task  involving  multiple  iterations  of  natural  language  descriptions, 
graphical  flow  descriptions  and  prototypical  user  interface  representations.  The 
resulting  artifact  is  in  an  essentially  static  representation  of  the  user’s  desires  in  an 
actionable  form  for  developers.  Reducing  the  time  to  create  operational  concepts, 
expanding  the  user-developer  bandwidth  and  extending  the  static  representation  to  a 
dynamic,  environment-aware  and  malleable  artifact  should  significantly  reduce  overall 
development  time  by  shortening  requirements  elicitation  and  reducing  rework  due  to 
misunderstanding.  There  is  little  ability  to  put  the  CONOPS  in  motion,  visually  observe 
behavior,  interact  with  the  analyst,  communicate  in  real  time,  and  develop  a  shared 
understanding  of  the  problem  or  mission,  and  likely  solution  approaches.  While  there 
are  complex  modeling  environments  that  allow  dynamic  modeling  (simulations),  they 
require  extensive  computer  programming  skill  and  significant  time  to  build  these 
simulations. 

Using  the  extensive  bodies  of  research  in  model  based  systems  engineering,  requirement 
elicitation,  mental  models  and  shared  mental  model  articulation,  negotiation  and 
decision  analysis  tools,  modeling  tools,  description  languages,  GUI  generators  and 
collaboration  environments;  our  objective  is  to  describe  a  graphical  and  interactive 
environment  to  model  a  concept  of  operations  in  support  of  agile  development  and  lean 
systems  engineering.  To  date,  however,  no  comprehensive  survey  of  existing  capabilities 
and  research  has  been  undertaken  with  this  specific  objective  in  mind.  It  is  anticipated 
there  are  a  number  of  technologies  and  classes  of  tools  that  might  be  useful  in  this 
research. 


1  http://www.sercuarc.org/ 
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2.2  Research  approach 

This  effort  exercised  several  threads  of  research  during  the  period  of  performance  with 
the  goals  of: 

•  Examining  core  and  customized  CONOPS  elements 

•  Mapping  critical  CONOPS  Process  Steps 

•  Developing  an  Agile  CONOPS  Process  Framework 

•  Identifying  current  and  emerging  technologies  that  could  serve  as  the  toolbox  for 
a  graphical  CONOPS  system 

•  Recommending  further  research 

We  began  with  a  survey  of  public,  academic,  and  conference  literature  to  understand  the 
breadth  and  depth  of  research  concerning  the  development  of  operational  concepts.  The 
results  of  that  survey  are  contained  in  the  annotated  bibliography  at  the  end  of  this 
report.  Additionally,  a  review  of  23  representative  strategic,  operational,  and  product¬ 
centric  CONOPS  was  conducted.  The  results  of  this  analysis  can  be  found  in  Section  3.0. 

Simultaneously,  we  interviewed  key  sources  and  subject  matter  experts  in  DoD  and 
commercial  enterprises  who  have  experience  developing  operational  concepts  for  their 
particular  industry  or  domain.  These  results  are  documented  in  Section  3.5. 

Based  on  the  research  and  interview  findings,  we  identified  five  key  challenges  to 
creating  CONOPS  efficiently  and  effectively:  value  proposition,  translation  of  concept 
decisions  into  system  attributes,  shared  mental  model  capability,  human  dimension, 
and  process  nonlinearity.  These  challenges  are  discussed  in  Section  3.6. 

In  Section  4.0,  we  address  these  challenges.  We  propose  a  three-stage  agile  CONOPS 
process  that  emphasizes  stakeholder  involvement  and  expedites  shared  mental  model 
development  (Section  4.1).  Our  notion  of  a  graphical  CONOPS  system  that  incorporates 
current  and  emerging  technologies  is  described  in  Section  4.2.  Next,  we  highlight  the 
salient  features  of  our  approach  (Section  4.3). 

Finally,  based  on  the  research  conducted  for  this  study,  we  make  recommendations  for 
future  research  that  is  necessary  to  bring  our  approach  to  fruition  in  Section  5.0. 


Contract  Number:  H98230-08-D-0171  DO  001,10  002  RT3  CONOPS 

Report  No.  SERC- 2009- TR- 003 
October  30,  2009 


UNCLASSIFIED 

11 


UNCLASSIFIED 


3  CONOPSAnalyss 


The  Concept  of  Operations  (CONOPS)  as  conceived  today  is  a  document  that  captures 
the  user’s  needs  and  vision  for  anything  being  conceptualized  for  the  purpose  of 
transforming  that  concept  into  reality.  The  following  sections  further  expand  on  that 
notion,  and  provide  detailed  analyses  of  representative  bodies  of  knowledge  on  the 
subject. 


3.1  DEFINmON 

A  Concept  of  Operations  (CONOPS)  is  a  document  describing  the  characteristics  of  a 
proposed  system  from  the  viewpoint  of  its  users.  It  is  used  to  communicate  the 
quantitative  and  qualitative  system  characteristics  to  all  stakeholders  and  serve  as  a 
basis  for  stakeholder  discussions  on  the  issue.  The  CONOPS  can  help  reach  a  “meeting 
of  the  minds”  before  the  requirements  process  begins.  It  often  conveys  a  clearer 
statement  of  intent  than  the  requirements  themselves. 

The  CONOPS  approach  provides  an  analysis  activity  and  a  document  that  bridges  the 
gap  between  the  user's  needs  and  visions  and  the  developer's  technical  specifications 
[IEEE1362].  In  addition,  the  CONOPS  document  provides  the  following: 

•  A  means  of  describing  a  user's  operational  needs  without  becoming  bogged  down 
in  detailed  technical  issues  that  shall  be  addressed  during  the  systems  analysis 
activity. 

•  A  mechanism  for  documenting  a  system's  characteristics  and  the  user's 
operational  needs  in  a  manner  that  can  be  verified  by  the  user  without  requiring 
any  technical  knowledge  beyond  that  required  to  perform  normal  job  functions. 

•  A  place  for  users  to  state  their  desires,  visions,  and  expectations  without 
requiring  the  provision  of  quantified,  testable  specifications.  For  example,  the 
users  could  express  their  need  for  a  "highly  reliable"  system,  and  their  reasons  for 
that  need,  without  having  to  produce  a  testable  reliability  requirement.  In  this 
case,  the  user's  need  for  "high  reliability"  might  be  stated  in  quantitative  terms  by 
the  buyer  prior  to  issuing  a  request  for  proposal  (RFP),  or  it  might  be  quantified 
by  the  developer  during  requirements  analysis.  In  any  case,  it  is  the  job  of  the 
buyer  and/or  the  developer  to  quantity  users'  needs. 

IEEE1362  further  states  that  a  CONOPS  is  a  mechanism  for  users  and  buyer(s)  to 
express  thoughts  and  concerns  on  possible  solution  strategies.  In  some  cases,  design 
constraints  dictate  particular  approaches.  In  other  cases,  there  may  be  a  variety  of 
acceptable  solution  strategies.  The  CONOPS  document  allows  users  and  buyer(s)  to 
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record  design  constraints,  provide  the  rationale  for  those  constraints,  and  indicate  the 
range  of  acceptable  solution  strategies. 


3.2  CO  NO  PS  Standards 

The  following  standards  are  used  by  different  industries  to  develop  CONOPS 
documents: 

•  ANSI/AIAA  G-043-1992  -  guide  from  American  National  Standards  Institute 

•  IEEE  1362-1998  -  IEEE  guide  for  CONOPS  document 

•  DI-IPSC-81430  -  DoD  data  item  description  for  CONOPS  document 

•  USDOT  Federal  Highway  Administration  CONOPS  Template 

Figure  1  is  the  ANSI  Operations  Concepts  Document  outline,  Figure  2  is  the  IEEE 
CONOPS  outline,  and  Figure  3  is  the  DoD  CONOPS  Elements.  As  can  be  seen,  while 
each  contains  similar  information,  there  are  differences  in  sections,  terms  and 
completeness  of  content.  For  instance,  while  the  ANSI  standard  calls  for  a  system 
architecture  to  be  part  of  the  CONOPS,  the  two  later  standards  -  IEEE  and  DoD  do  not. 
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1.  Scope.  System  identification, 

8.  Operational  Scenarios.  Detailed  sequences  of  user, 

purpose,  and  overview.  Contents, 

system,  and  environmental  events: 

intention,  and  audience  for  the 

-  Normal  conditions  -  "Stress"  conditions 

operational  concept  document 

-  Failure  Event  -  Maintenance  mode 

-  Handling  anomalies 

2.  Referenced  Documents 


3.  User-Oriented  Operational  Description. 

How  mission  accomplished:  strategies,  tactics, 
policies,  constraints.  Who  users  are  and  what 
the  users  do. 

-  When  and  in  what  order  operations  take  place 

-  Personnel  profile;  organizational  structure 

-  Personnel  interactions;  activities 

-  Operational  process  models:  sequence, 
interrelationships 


4.  Operational  Needs.  Mission  and  personal 
needs  that  drive  the  requirements  for  the 
system 

5.  System  Overview.  Scope;  users;  interfaces; 
states  and  modes;  capabilities;  goals  and 
objectives;  system  architecture 


6.  Operational  Environment 


7.  Support  Environment 


Figure  1:  ANSI/AIAA  Outline  IbrOpeiations  Concept  Document 
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Figure  3:  DI-IPSC-81430CONOPS  Elements  (DoD) 


Contract  Number:  H98230-08-D-0171  DO  001,10  002RT3CONOPS 

Report  No.  SERC- 2009- TR- 003 
October  30,  2009 

UNCLASSIFIED 

16 


UNCLASSIFIED 


3.3  TVpesofCONOPS 


Normally,  a  CONOPS  falls  into  one  of  three  categories,  as  listed  below: 


•  Development  of  a  New  System 

•  Modification/  Upgrade/  Change  to  Existing  System/Product 

•  Operational  Strategy  (which  may  also  include  end  of  life  activities) 


3.4  CO  NO  PS  Contents 


To  better  understand  the  detailed  contents  of  CONOPS,  we  looked  at  23  different 
CONOPS  documents  for  real  world  systems  developed  by  a  variety  of  government  and 
private  sector  institutions  for  new  systems,  modifications  to  existing  systems  and  for 
mapping  out  operational  strategies.  These  documents  represented  strategic, 
operational,  and  product-centric  perspectives.  The  goal  was  to  see  to  what  extent  each  of 
the  CONOPS  documents  had  incorporated  the  entirety  of  the  CONOPS  elements  and  to 
what  extent  different  elements  had  been  omitted.  We  examined  12  CONOPS  documents 
at  a  strategic  element  level  (as  identified  in  Appendix  E)  and  another  13  CONOPS 
documents  at  detailed  step  levels.  We  also  looked  at  the  process,  where  documented, 
through  which  the  CONOPS  was  created.  Table  1  represents  the  industries  and  sectors  of 
the  studied  CONOPS. 


lable  1:  Representative  Industries  for  Mined  CONOPS 


Software  Engineering 

Aerospace 

Defense 

Transportation 

Education 


IT  Systems 
Weather  Systems 
General/ Cross-Industry 


Communications 

Pharmaceutical 


The  names  of  the  individual  documents  are  shown  in  boxes  1,  2-8  in  Figure  1.  By 
mapping  the  sections  in  each  actual  CONOPS  document  to  those  in  the  standards,  the 
most  common  sections  for  describing  a  concept  of  operations  were  identified. 
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3.4. 1C  O  NO  PS  C  OMPARISO  N 


Table  2:  CONOPS  Comparison  Matrix 
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Key  of  Industries  SWE-Software  Engineering 
IT  -  Info  Technology 


AERO  -  Aerospace 
WEA  -  Weather 


DEF  -  Defense 
3  -  General/Cross-Industr 


FRAN  -  Transportation 
PHAR  -  Pharmecutical 


EDU  -  Education 
OMM  -  Communicatior 


Key  of  Examples 

1.001  TRAN  US  Department  of  Transportation:  Federal  Highway  Administration,  "Concept  of  Operations," 

1.003  SWE  IEEE,  "IEEE  Guide  for  Information  Technology  —  System  Definition  —  Concept  of  Operations  (ConOps)  Document,"  vol.  IEEE  Std  1362™-1998  (R2007),  1998. 

1.014  AERO  American  Institute  of  Aeronautics  and  Astronautics,  "ANSI/AIAA  Guide  for  the  Preparation  of  Operational  Concept  Documents,"  vol.  ANSI/AIAA  G-043-1992,  1993. 

1.040  DEF  Department  of  Defense,  "Operation  concept  description,"  Washington  DC,  Tech.  Rep.  DI-IPSC-81430,  1994. 

2.005  EDU  J.  Ring  and  A.  W.  Wymore,  "Overview  of  a  CONOPS  for  an  SE  education  community,"  in  Tenth  Annual  International  Symposium  of  the  International  Council  on  Systems  Engineering,  2000, 

2.028  AERO  A.  Haraldsdottir,  R.  W.  Schwab  and  M.  S.  Alcabin,  "Air  traffic  management  capacity-  driven  operational  concept  through  2015,"  in  2nd  USA/Europe  Air  Traffic  Management  R&D  Seminar,  1998, 

2.035  DEF  US  Air  Force,  "Air  force  smart  operations  for  the  21st  century  CONOPS," 

2.033  IT  Integrated  Computer  Engineering,  "Concept  of  operations  for  electronic  records  archives,"  Electronic  Records  Archives  -  Program  Management  Office,  2004. 

2.022  SWE  Software  Engineering  Institute,  "Concept  of  operations  for  the  CMMI,"  Carnegie  Mellon  University,  January  15,  2001,  2001. 

2.029  AERO  K.  Johnston  and  J.  Ladd,  "A  concept  of  operations  for  an  integrated  weather  forecast  process  to  support  the  national  airspace  system,"  in  11th  Conference  on  Aviation,  Range,  and  Aerospace,  2004, 

2.003  SWE/G  The  Boeing  Company  and  Lockheed  Martin  Corporation,  "CONOPS  for  the  systems  and  software  test  track," 

2.012  PHAR/IT  G.  Karlsson  and  S.  Wauthier.  Operational  concept  document:  A  process  based  integration  of  production  IT  solutions  in  a  pharmaceutical  plant.  Presented  at  World  Batch  Forum  North  American  Conference. 

2.002  DEF  US  Marine  Corps,  "Common  logistics  command  and  control  system  concept  of  operations,"  April  19,  2004,  2004. 

2.034  DEF  F.  W.  Lacroix,  R.  W.  Button,  S.  E.  Johnson,  J.  Chiesa  and  J.  R.  Wise,  "A  concept  of  operations  for  a  new  deep-diving  submarine,"  RAND:  National  Defense  Research  Institute,  2002. 

2.011  TRAN  I.  Mixon/Hill,  "StarTran  automated  vehicle  location  system  concept  of  operations,"  Lincoln,  NE,  November  2005,  2005 

2.001  WEA  K.  Lynott,  "NOAA's  national  weather  service  concept  of  operations  river  forecast  center  (RFC)  analysis  and  gridded  forecast  editor  improvement,"  NOAA,  July  19,  2005,  2005. 

2.006  COMM  Maine  State  Office  of  Information  Technology  and  Maine  Emergency  Management  Agency,  "State  of  maine  ConOps  for  incident  communications  interoperability,"  State  of  Maine,  March  19,  2007,  2007 

2.020  G  World  Intellectual  Property  Organization,  "Concept  of  operations  for  the  reformed  IPC,"  Tech.  Rep.  IPC/CE/36/11,  Annex  X,  February  18,  2005,  2005. 

2.032  AERO  S.  Darr,  "NASA  aviation  safety  &  security  program  (AvSSP)  concept  of  operation  (CONOPS)  for  health  monitoring  and  maintenance  systems  products,"  ICF  Consulting,  Tech.  Rep.  NIA  Report  #  2006-04,  September  2005,  2005. 

2.031  DEF  H.  Gilibert  and  R.  Thevenot,  "CONOPS  of  HALE  UTA  in  an  InfraRed  early  warning  mission  for  theater  missiles  defense,"  in  AGARD  MSP  Symposium  on  "System  Design  Considerations  for  Unmunned  Tactical  Aircraft  (LITA)",  1997 

2.015  IT  Booz  Allen  Hamilton,  "Concept  of  operations  (CONOPS)  environmental  protection  agency  financial  system  modernization  project,"  US  Environmental  Protection  Agency,  October  26,  2005,  2005. 

2.036  IT  Grants.gov,  "Concept  of  operations  grants.gov  system,"  April  17,  2007,  2007. 

2.037  DEF  US  Air  Force,  "Force  development  CONOPS  civilian,"  January  2006,  2006. 

2.038  WEA  NOAA,  "Integrated  ocean  observing  system  data  management  and  communications  concept  of  operations,"  October  2008,  2008. 

2.039  G  UN  System  Influenza  Coordinator  and  Pandemic  Influenza  Contingency  Team,  "Concept  of  operations  for  the  UN  system  in  an  influenza  pandemic,"  September  19,  2008,  2008. 

2.040  WEA/AERC  NOAA  and  NASA,  "Geostationary  operational  environmental  satellite  (GOES)  concept  of  operations,"  February  2008,  2008. 

2.041  G  Business  Transformation  Agency,  "Concept  of  operations  for  business  enterprise  architecture  (BEA)  requirements,"  Septmeber  14,  2007,  2007. 
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3.4.2  C  ontents  Analysis 

Analysis  of  the  CONOPS  comparison  matrix  in  Table  2  was  conducted  in  four  stages. 
First,  the  matrix  allowed  for  determination  of  which  CONOPS  elements  were  most  and 
least  commonly  used  among  the  collected  examples.  Next,  a  comparison  was  done  to 
identify  the  common  traits  of  CONOPS  with  the  industry  as  a  variable.  This 
identification  led  to  an  investigation  into  the  adherence  of  the  examples  to  existing 
standards,  analyzed  both  independently  and  by  industry.  Finally,  the  last  useful  analysis 
made  possible  by  this  matrix  involved  evaluating  the  format  of  the  CONOPS  examples. 

The  sample  size  of  this  evaluation  was  somewhat  small,  and  any  conclusions  should  be 
validated  with  a  larger  number  of  CONOPS  examples  from  each  industry  and  CONOPS 
type.  At  first  glance,  it  appeared  as  if  there  were  few  patterns  to  be  extracted  from  the 
data.  It  seemed  as  if  the  contents  of  each  CONOPS  example  were  largely  based  on  the 
choices  of  the  author  or  the  organization  for  which  they  were  prepared.  However, 
placing  the  matrix  results  into  a  graphical  form  provided  a  visual  and  side-by-side 
comparison,  which  enabled  some  conclusions  to  be  reached. 


3A2.1  CONOPS  Element  Structure 

One  goal  was  to  determine  which  elements  or  sections  were  the  most  commonly  used 
across  the  surveyed  CONOPS  documents.  Table  3  shows  which  15  sections  appeared  in 
at  least  half  of  the  documents. 

Table  3:  Most  Commonly  Used  CONOPS  Elements 


Element 

Occurrences 

%  of  Total 

Proposed  System 

Description  (Concept) 

23 

100.00 

Proposed  System 

System  Capabilities 

20 

86.96 

Proposed  System 

Objectives 

18 

78.26 

Proposed  System 

Operational  Environment 

16 

69-57 

Proposed  System 

System  Interfaces 

16 

69-57 

Proposed  System 

Personnel  Activities 

14 

60.87 

Current  System 

Background 

13 

56.52 

Proposed  System 

Personnel  Profile 

13 

56.52 

Proposed  System 

Personnel  Type 

13 

56.52 

Changes 

Mission  Needs 

12 

52.17 

Proposed  System 

Scope 

12 

52.17 

Proposed  System 

Operational  Policies 

12 

52.17 

Proposed  System 

High  level  Requirements 

12 

52.17 

Analysis 

Summary  of  Improvements 

12 

52.17 

Miscellaneous 

Operational  Scenarios 

12 

52.17 

23  Total  CONOPS  Examples  studied 
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Since  the  CONOPS  documents  represent  how  a  new  (or  upgrade  of  an  existing)  system 
will  be  used  in  an  operational  context,  it  seems  obvious  that  the  proposed  system 
description  would  be  found  in  all  twenty-three  of  the  CONOPS  examined,  as  was  found 
to  be  true.  Also  appearing  in  the  vast  majority  of  the  surveyed  CONOPS  were  the 
proposed  system  objectives  and  scope,  important  for  assisting  in  managing  scope  creep. 
The  majority  of  CONOPS  went  into  some  level  of  detail  when  examining  the  problem 
space.  It  would  also  be  natural  to  believe  that  the  criticality  of  early  definition  of  a 
proposed  system’s  capabilities,  interface  and  operational  environment  would  also  be 
part  of  every  CONOPS,  but  the  analysis  shows  that  these  items  only  appeared  in  75%  of 
the  CONOPS  examples. 

More  troublesome,  almost  70%  of  the  CONOPS  (69.57%)  researched  did  not  actually  list 
or  identify  specific  mission  needs.  This  becomes  a  challenge  to  the  systems  developers 
who  have  the  responsibility  to  build  a  system  that  satisfies  those  specific  needs.  Also 
noted  was  that  of  the  twenty-three  CONOPS  evaluated,  nearly  a  third  had  no  description 
of  the  background  or  context  of  the  current  system  or  situation.  Each  of  the  CONOPS 
standards  used  as  guidance  in  this  study  calls  for  the  system  background  to  be  laid  out 
prior  to  describing  the  proposed  system.  The  standards  also  recommend  that  a  full 
description  of  the  current  system/situation  be  made  available  in  the  CONOPS 
document,  yet  almost  half  of  the  examined  examples  neglected  to  include  such  a 
description.  Generally  speaking,  there  was  a  shortage  of  information  relating  to  the 
current  system/situation.  In  over  50%  of  the  CONOPS  examined,  the  provided  current 
system  background  was  a  list  of  current  system  components  in  use.  Table  4  shows  the 
least  often  used  elements,  found  in  less  than  25%  of  the  CONOPS  examples. 

lable  4:  LeastCommonly  Used  CONOPS  Elements 


Element 

Occurrences 

%  of  Total 

Current  System 

Non-functional  Attributes 

1 

4-35 

Miscellaneous 

Stakeholder  Assessment 

1 

4-35 

Current  System 

Performance  Characteristics 

2 

8.70 

Changes 

Considered  but  not  included 

2 

8.70 

Current  System 

Organizational  Structure 

3 

13.04 

Current  System 

System  Interfaces 

3 

13.04 

Miscellaneous 

Associated  Risks 

3 

13.04 

Current  System 

Modes  of  Operation 

4 

17-39 

Current  System 

Personnel  Interfaces 

4 

17-39 

Changes 

Personnel  Needs 

4 

17-39 

23  Total  CONOPS  Examples  studied 

Of  the  ten  elements  listed  in  Table  4,  six  of  them  are  related  to  the  current 
system/situation.  Some  of  these,  including  personnel  and  system  interfaces  and  modes 
of  operation  are  significant  to  the  operation  of  a  system.  Another  particularly  weak  area 
not  addressed  in  most  of  the  CONOPS  was  stakeholder  assessment.  While  system 
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stakeholders  include  those  who  will  operate  and  support  the  system  (in  Table  2,  they  are 
grouped  together  as  personnel  and  support),  little  attention  was  paid  to  other 
stakeholders  who  do  not  directly  interact  with  the  system,  including  acquisitions  staff, 
and  government  and  regulatory  agencies.  Stakeholder  assessment  was  not  included  in 
any  of  the  standards  and  is  typically  addressed  throughout  the  early  stages  of 
development,  which  perhaps  explains  the  reason  why  it  was  found  in  only  one  of  the 
examples. 

Yet  another  weak  area  was  the  changes/alternatives  that  were  considered  but  not 
included  in  the  proposed  system.  The  neglect  of  these  elements  may  highlight  the 
common  systems  engineering  error  of  not  keeping  the  problem  and  solution  spaces 
separate.  Finally,  less  than  20%  of  the  CONOPS  examples  identified  the  associated  risks 
of  the  system  and  its  development.  This  was  unexpected  given  the  fact  that  many  of  the 
examples  were  written  by/for  government  agencies,  which  are  typically  highly 
concerned  with  thorough  risk  analysis. 


3A2.2  CO  NO  PS  Elements  by  Industry 

The  next  layer  of  analysis  made  of  this  data  was  examination  of  common  CONOPS  traits 
by  industry.  To  facilitate  this  analysis,  three  general  industries  were  identified,  (defense, 
software  and  aerospace)  with  each  containing  five  of  the  CONOPS  examples  (Figures  4, 
5,  and  6).  This  division  ended  up  excluding  eight  examples  that  did  not  fit  into  these 
categories,  but  made  for  easier  evaluation  of  the  data.  Since  exactly  five  examples  from 
each  industry  were  used,  the  scales  are  the  same  in  each  chart,  and  the  height  of  the  bars 
in  the  following  graphs  represents  the  number  of  specific  CONOPS  examples  which 
contained  the  elements  listed  in  the  x-axis.  The  dotted  red  lines  correspond  to  the  topic 
boundaries  found  on  Table  2. 

Certain  observations  can  be  made  regarding  the  industry  specific  CONOPS  examples.  In 
the  software  industry,  most  of  the  CONOPS  examples  did  a  thorough  evaluation  of  the 
proposed  system,  yet  there  was  little  effort  expended  in  describing  the  current  system. 
Contrast  this  to  the  aerospace  industry,  where  the  CONOPS  contained  more  information 
related  to  the  current  system. 

Typically,  the  CONOPS  examined  were  for  new  systems  that  did  not  exist  or  systems 
that  were  meant  to  completely  replace  existing  systems.  In  the  aerospace  industry,  there 
is  much  more  dependence  on  legacy  systems  and  component  reuse,  which  would  require 
a  deeper  understanding  of  the  current  system.  The  defense  industry  is  similar  to 
aerospace  in  that  legacy  systems  are  dominant  platforms  from  which  to  develop  new 
systems.  But  strangely,  similar  to  software  systems,  defense  system  CONOPS  provided 
less  analysis  of  current  systems/situations.  This  is  an  area  that  would  benefit  greatly 
from  evaluation  of  additional  examples,  as  well  as  access  to  classified  CONOPS. 
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Defense  industry  CONOPS  examples  also  appear  to  have  scored  poorly  in  evaluating 
risks  associated  with  system  development,  as  well  as  other  general  impacts  of  system 
development  and  implementation.  As  mentioned  previously,  this  result  is  interesting  in 
respect  to  government  entities  in  general  and  the  defense  industry  specifically,  as  both 
are  particularly  concerned  with  risk  analysis  and  mitigation.  Another  observation 
regarding  industry  specific  CONOPS  was  in  the  level  of  detail  used  to  examine  use  cases, 
operational  scenarios  and  modes  of  operation.  In  the  software  industry,  these  three 
elements  ranked  relatively  high,  with  aerospace  CONOPS  giving  less  detail  and  the 
defense  CONOPS  giving  almost  no  detail  for  these  aspects.  The  reason  for  this  disparity 
is  unknown,  and  perhaps  investigation  of  more  CONOPS  in  these  industries  will  provide 
some  further  insights. 
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Figure  4:  CONOPS  Elements  in  the  Defense  Industry 
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Figure  5:  CONOPS  Elements  in  the  Software  Industry 
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It  appears  that  the  aerospace  and  software  industry  CONOPS  have  the  most  in  common 
while  the  defense  industry  differs  from  both  of  them  by  many  factors.  Another 
observation  is  that  the  aerospace  industry  utilizes  many  of  the  categories  at  a  higher  rate 
than  the  other  two  industries.  There  has  been  nothing  in  our  research  that  would 
explain  this  and  nothing  in  the  standards  that  would  cause  this  to  happen. 

Quantitatively,  since  there  were  equal  numbers  of  examples  and  elements  for  each 
industry,  the  total  “scores”  were  added  up  to  give  an  impression  of  which  industries 
have  the  most  inclusive  CONOPS  (in  relation  to  the  specific  element  evaluated).  While 
this  is  not  a  measure  of  the  quality  of  the  CONOPS  examples,  since  it  took  no  account  of 
what  was  written,  it  is  a  measure  of  the  inclusivity  of  information.  The  score  was 
calculated  by  adding  up  the  number  of  CONOPS  examples  that  include  each  element. 
Using  this  approach,  the  maximum  score  for  each  industry  would  be  265  (53  elements 
[excluding  those  involving  format]  *  maximum  score  of  5  out  of  5  =  265).  The  software 
and  aerospace  industries  both  scored  136  (average  score  of  2.6  per  element  [out  of  4 
maximum])  and  the  defense  industry  scored  90  (average  of  1.7  per  element).  Again, 
although  these  scores  are  not  an  effective  measure  of  the  quality  of  CONOPS,  they  may 
be  useful  in  understanding  how  the  CONOPS  match  up  against  the  standards.  This  is 
summarized  in  Table  5. 

Table  5:  CONOPS  Completeness 


Software 

CONOPS 

Aerospace 

CONOPS 

DoD 

CONOPS 

Number  of  CONOPS  Elements 

53 

53 

53 

Average  use  per  element 

2.6 

2.6 

1-7 

Maximum  Score  Possible 

265 

265 

265 

Completeness  Score 

136 

136 

90 

3A2.3  CONOPS  Elements:  Standards  versus  Examples 

When  looking  at  the  18  categories  that  can  be  found  in  all  four  CONOPS  standards  only 
nine  (50%  of  the  18  categories)  where  documented  at  a  rate  of  50%  or  more  in  the 
reviewed  CONOPS  (Table  6).  Using  the  logic  that  those  18  categories  must  be  the  most 
critical  since  they  are  referenced  in  all  four  standards  one  might  expect  to  find  all  18 
categories  utilized  in  the  majority  of  CONOPS  reviewed.  However,  the  four  least  utilized 
categories  out  of  the  18  are  all  related  to  personnel  (Personnel  Needs,  Personnel 
Activities,  Personnel  Types,  and  Personnel  Profiles).  These  are  used  at  a  rate  of  only 
22%  or  less  in  the  CONOPS  reviewed.  It  is  also  interesting  to  note  that  5  out  of  the  9 
least  utilized  categories  are  related  to  personnel  attributes  of  the  proposed  system. 

Since  the  elements  for  the  matrix  in  Table  2  were  chosen  using  the  four  leading 
CONOPS  standards,  it  was  expected  that  each  example  and  each  industry  as  a  group 
would  contain  elements  that  conform  to  their  relevant  standard.  Contrary  to 
expectation,  most  of  the  examples  in  this  study  followed  the  format  of  one  of  the  IEEE 
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and  AIAA  standards.  The  AIAA  standard  is  fairly  antiquated,  and  the  Federal  Highway 
Administration  standard  tends  to  be  highway  transportation  specific,  therefore  this 
leads  authors  of  non-defense  CONOPS  documents  to  use  the  more  inclusive  and 
adaptable  IEEE  standard  as  a  guide  to  CONOPS  development.  The  CONOPS  examples 
used  for  this  study  varied  in  their  degree  of  conformance  to  the  IEEE  standard,  but  no 
industry  group  of  CONOPS  conformed  complete  to  any  single  standard. 


Table  6:  Comparative  Analysis  with  CO  NOP  Standards 


Element 

%  of  Total 

AIAA 

DoT 

DoD 

IEEE 

Description  (Concept) 

100.00 

1 

1 

1 

1 

System  Capabilities 

86.96 

1 

1 

1 

1 

Objectives 

78.26 

1 

1 

1 

1 

Operational  Environment 

69-57 

1 

1 

1 

1 

System  Interfaces 

69-57 

1 

1 

1 

1 

Personnel  Activities 

60.87 

1 

l 

1 

l 

Background 

56.52 

1 

l 

1 

l 

Mission  Needs 

52.17 

1 

l 

1 

1 

Scope 

52.17 

1 

1 

1 

1 

Support  Environment 

47-83 

1 

1 

1 

1 

Modes  of  Operation 

47-83 

1 

l 

1 

l 

High  Level  Architecture 

43-48 

1 

1 

1 

1 

Personnel  Interfaces 

39-13 

1 

l 

1 

l 

Organizational  Structure 

34-78 

1 

1 

1 

l 

Personnel  Profile 

21-74 

1 

1 

1 

1 

Personnel  Activities 

21-74 

1 

l 

1 

l 

Personnel  Type 

21-74 

1 

l 

1 

l 

Personnel  Needs 

17-39 

1 

1 

1 

1 

23  Total  CONOPS  Examples  studied 

3A2.4  CO  NO  PS  Graphical  Elements 

A  final  analysis  of  the  surveyed  CONOPS  was  in  relation  to  the  format  of  CONOPS 
documents.  This  was  of  specific  interest  to  this  research  task,  which  is  aimed  at  creating 
a  visual  representation  of  CONOPS  documentation.  Table  7  shows  that  of  the  twenty- 
three  CONOPS  examined,  close  to  75%  were  made  up  of  a  mixture  of  text  and  graphics, 
just  over  25%  had  text  only,  and  none  of  them  were  presented  with  only  graphics. 
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Table  7:  CONOPSGraphical  Bements 


F  ormat/ Graphics 

Occurrences 

%  of  Total 

Text  Only 

6 

26.09 

Text  and  Graphics 

17 

73-91 

Graphics  Only 

0 

0.00 

Flowcharts 

10 

43-48 

Organizational  Charts 

5 

21.74 

Context  Diagram 

5 

21.74 

Formal  SE  Diagrams 

2 

8.70 

Survey/Data  Graphics 

3 

13.04 

High  Level  Architecture 

11 

47-83 

23  Total  CONOPS  Examples  studied 

Of  the  seventeen  examples  that  had  utilized  graphics,  ten  included  some  form  of  data  or 
process  flowchart  or  high-level  architecture  graphic.  However,  only  two  of  them 
incorporated  a  formal  systems  engineering  graphic.  These  results  highlight  that  while 
there  may  be  a  desire  to  incorporate  graphics  to  improve  clarity,  there  is  a  lack  of 
structure  when  choosing  what  types  of  graphics  to  include. 


3A2.5  Potential  Sourc  es  of  Sounc  e  Data  Bias 

This  analysis  was  performed  based  on  a  collection  of  CONOPS  which  were  free  and 
accessible  via  the  Internet.  This  may  bias  the  analysis  in  following  three  directions,  as  a 
result  of  the  data: 

1.  Private  companies  are  typically  interested  in  protecting  any  edge  they  have  over 
their  competition,  therefore  they  are  frequently  unwilling  to  release  information 
related  to  proprietary  products  and  processes. 

2.  Government  entities  restrict  the  release  of  sensitive  information  for  reasons  of 
national  security,  therefore  reducing  in  number  what  should  have  been  the 
largest  pool  from  which  to  select  examples. 

3.  As  can  be  seen  by  examining  the  last  column  of  the  matrix  in  Table  2,  many  of  the 
CONOPS  examples  are  rather  short  in  length.  Due  to  the  two  points  listed  above, 
companies  and  government  entities  often  allow  the  release  of  certain  information 
from  CONOPS  documents  to  be  used  in  shorter,  less  detailed  conference  and 
white  papers.  These  CONOPS  may  not  only  exclude  sensitive  information,  but 
may  also  exclude  some  of  the  elements  that  this  matrix  has  been  examining, 
leading  to  misrepresentation  in  the  analysis. 
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3.4.3  C  O  NO  PS  Analysis  C  onc  uusions 

Summarizing  the  CONOPS  analysis,  of  the  issues  identified,  the  following  key  issues 
may  be  significant  in  developing  an  agile  approach  for  graphical  CONOPS: 

•  Less  than  75%  of  the  CONOPS  researched  actually  list  or  identify  specific  mission 
needs. 

•  Nearly  a  third  had  no  description  of  the  background  or  context  of  the  current 
system  or  situation. 

•  There  was  a  shortage  of  information  relating  to  the  current  system/situation. 

•  Little  attention  was  paid  to  other  stakeholders  who  do  not  directly  interact  with 
the  system,  including  acquisitions  staff,  and  government  and  regulatory  agencies. 

•  Stakeholder  assessment  was  not  included  in  any  of  the  standards  and  is  typically 
addressed  later  throughout  the  early  stages  of  development. 

•  Less  than  20%  of  the  CONOPS  examples  identified  associated  risks  of  the  system 
and  its  development. 

•  The  four  least  utilized  categories  out  of  the  18  are  all  related  to  personnel 
(Personnel  Needs,  Personnel  Activities,  Personnel  Types,  and  Personnel  Profiles). 


3.5  CurreisttCO  NO  PS  Creation  Process 

Another  task  performed  was  to  research  published  process  documents  and  conduct 
interviews  across  domains  to  understand  how  the  CONOPS  process  plays  out  in  the 
field.  While  many  organizations  do  not  call  the  resulting  product  a  “concept  of 
operations”,  the  spirit  of  activities  is  the  same.  The  following  sections  detail  one  DoD 
process,  and  recounts  the  results  of  several  industry  interviews. 

We  found  that  CONOPS  documents  that  adhered  to  all  the  different  steps  took  as  long 
as  30  months  to  produce.  In  other  cases,  when  the  bare  minimum  elements  were 
selected,  CONOPS  documents  were  finished  within  3  months.  In  most  cases  the 
CONOPS  development  process  was  performed  by  a  core  CONOPS  team  and  the  draft 
was  sent  out  for  review  to  the  relevant  stakeholders.  It  was  found  that  the  required  time 
to  gain  consensus  through  back  and  forth  discussions  among  the  various  stakeholders  in 
the  organization  consumed  the  bulk  of  the  CONOPS  development  effort.  Also  the  text- 
based  nature  of  the  CONOPS  makes  editing  of  CONOPS  documents  a  time-consuming 
and  challenging  process.  In  most  cases  the  CONOPS  seems  to  have  been  produced  only 
due  to  documentation  requirements  rather  than  as  a  strategic/tactical  system  planning 
tool.  This  defies  the  original  purpose  of  the  CONOPS  which  is  to  mediate  between  user 
and  developer  communities  and  other  stakeholders  in  a  way  that  a  system  can  be 
designed  holistically  and  in  an  integrated  fashion.  We  present  the  Air  Force  Space 
Command  CONOPS  process  as  the  representative/traditional  approach  used  by  most 
organizations  today. 
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3.5.1Air  Forc  e  Spac  e  Command  Proc  ess 

The  Air  Force  space  command  (AFSPC)  CONOPS  creation  process  is  shown  in  Figure  7. 
This  is  a  very  traditional  view  and  seems  representative  of  government  agencies  as  well 
as  large  DoD  contractors.  The  flow  says  nothing  about  the  content  creation  of  the 
CONOPS,  just  the  CONOPS  artifact  creation  process. 

The  AFSPC  states  that  some  CONOPS  are  created  to  support  administrative  processes, 
and  other  are  created  for  programs.  They  use  the  CONOPS  to  provide  vision  for  the 
directorate  office,  and  it  is  intended  to  describe  how  the  system  or  capabilities  will  be 
operated  and  utilized  by  the  directorate. 

The  CONOPS  is  used  for  many  purposes  by  the  AFSPC.  Some  of  those  uses  include: 

•  Headquarters  extracts  operational  requirements  from  the  CONOPS  and  uses  it 
as  a  framework  for  developing  their  Operational  Requirements  Document 
(ORD). 

•  Other  directorates  may  use  the  CONOPS  for  developing  their  internal 
documents. 

•  The  air  wings  use  the  CONOPS  when  developing  their  concept  of  employment 
(COE)  and  may  use  the  CONOPS  for  Operation  Plan  (OPLAN)  development. 

•  Agencies  outside  the  command  also  have  a  vested  interest  in  the  CONOPS. 

•  The  system  program  offices  (SPO)  use  the  CONOPS  as  guidance  to  direct  the 
contractors  in  system  architecture  development  as  part  of  the  acquisition 
process. 

•  Contractors  use  the  CONOPS  as  guidance  for  developing  internal  architectural 
documents  to  meet  requirements. 

•  The  Air  Force  Operational  Test  and  Evaluation  Center  (AFOTEC)  uses  the 
CONOPS  when  conducting  their  operational  assessment  (OA)  and  as  part  of  the 
operational  test  and  evaluation  (OT&E)  concept  development. 

In  general,  the  CONOPS  provides  guidance  to  those  users  requiring  direction  and/or 
information  on  developing  their  own  documents  [AFSPCI 10-606]. 

Figure  8  shows  how  the  AFSPC  takes  the  flow  from  Figure  7,  and  translates  it  into  a 
notional  schedule.  The  process  is  extremely  linear,  and  the  example  translates  into  a  6- 
month  event. 
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Figure  7:  Air  Force  Space  Command  CONO PS  Development  Process 
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Figure  8:  Air  Force  Space  Command  CO  NO  PS  Process  Translated  into  a  Schedule 
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3.5.2  Examples  of  Key  Practitioners  in  Different  Do  mains 

In  the  private  sector,  a  CONOPS2  is  not  usually  created  in  a  formal  document  at  the 
initiation  of  a  program,  but  rather  evolves  over  the  program  lifecycle  either  through 
tribal  knowledge  within  the  project  organization,  or  piecemeal  in  a  number  of  product 
and  project  related  documents.  A  formal  CONOPS  is  usually  written  late  in  the 
development  cycle  after  a  system  is  up  and  running  and  just  before  initial  deployments 
(often  Beta  quality  products)  are  made.  This  CONOPS  is  usually  written  as  a  transfer  of 
knowledge  between  engineering,  product  management  and  the  service  organization 
which  is  responsible  for  the  installation  and  maintenance  of  the  system.  One  project 
phasing  approach  used  in  the  computer  industry  consists  of  internal  alpha  releases  of 
products  to  test  functionality  and  performance;  followed  by  limited  external  Beta 
releases  that  test  the  manufacturing,  delivery  and  installation  processes;  followed  by 
ramped  up  full  scale  production.  Since  a  formal  CONOPS  is  not  usually  created  and 
validated  until  late  in  the  development  cycle,  operational  issues  are  often  found  late. 

The  artifacts  from  CONOPS  work  that  are  completed  early  in  the  development  cycle 
often  find  their  way  into  Product  Requirements  documents  or  Product  Architecture  and 
Design  documents,  often  as  a  high-level  overview  description  of  operation  primarily  to 
provide  context  for  the  product,  architecture  or  design  details.  An  evolving  view  of  the 
operation  of  the  system  is  usually  developed  by  interactive  discussions  between  the 
architects,  designers  and  verifications/ validation  teams.  Occasionally  this  information 
is  documented  in  development  and  test  plans.  In  any  case,  the  CONOPS  is  generally  an 
informal,  evolving  set  of  specifications.  Disconnects  between  the  evolving  system 
operation  and  the  original  intent  of  the  product  marketing  organization,  or  business 
team  occur  and  can  be  quite  expensive  in  terms  of  product  delay,  product  functionality 
and  quality  issues,  and  development  inefficiencies. 

The  following  are  a  few  examples  of  programs  with  widely  varying  results.  It  should  be 
noted  that  a  formal  CONOPS  was  not  created  in  any  of  these  cases,  but  rather  shared 
mental  models  about  the  operation  were  created  through  discussion  and  disseminated 
in  a  number  of  different  documents. 


2  Note,  the  term  ‘CONOPS’  is  not  used  in  the  private  sector,  but  is  used  here  to  avoid  confusion. 
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3.5.2.1  Sun  Microsystems  "Serengeti”  Server  Development 

In  2001,  Sun  unveiled  a  newline  of  mid-range  servers,  code  named  “Serengeti”,  based 
on  the  UltraSPARC  III  microprocessor.  This  included  8-way,  12-way  and  24-way  rack 
mounted  systems.  This  was  very  much  an  engineering  driven  development  process  and 
the  product  evolved  based  largely  on  technical  tradeoffs  rather  than  value  attributes. 
While  the  system  did  not  meet  its  aggressive  schedule,  it  was  technically  a  sound 
product.  Unfortunately,  none  of  these  systems  had  the  same  form  factor  as  the  hugely 
successful  14 -way  Enterprise  4000  (introduced  in  1996)  and  the  4500  (introduced  in 
1999).  As  a  result,  Sun  lost  the  opportunity  to  capitalize  on  an  extremely  large  upgrade 
opportunity.  While  attempts  were  made  to  create  a  server  with  the  form  factor  of  the 
4000  and  4500  using  Serengeti  parts,  this  was  an  effort  that  came  too  late.  In  this  case, 
not  having  a  concept  of  operations  which  included  the  value  of  a  common  form  factor 
for  the  upgrade  of  systems  resulted  in  a  dramatic  loss  of  value  for  the  delivered  system. 


3. 5. 2. 2  Sun  Microsystems  “Eagle"  Server  Development 

In  an  attempt  to  avoid  the  engineering  driven  issues  of  Serengeti,  the  server 
development  code-named  “Eagle”,  based  on  the  UltraSPARC  V  microprocessor,  and 
incorporated  the  development  of  a  Product  Requirements  Document  into  the 
development  process.  Unfortunately,  this  was  a  long  and  protracted  process  due  to  a 
large  extent  on  conflicts  of  interests  within  the  organization.  In  particular,  there  was  a 
lack  of  clarity  on  how  value  would  be  created  in  the  market,  and  an  unwillingness  to 
make  tradeoffs  which  depended  on  this  knowledge.  In  addition,  those  who  were 
responsible  for  creating  the  requirements  did  not  have  the  authority  to  independently 
arbitrate  on  behalf  of  their  host  functional  organizations.  The  difficulty  in  finding 
available  time  from  those  with  the  appropriate  level  of  authority  resulted  in  delays  in 
meetings  and  interactive  decision  making.  As  a  result,  decisions  were  made  very  slowly, 
if  at  all.  The  net  result  was  that  after  significant  delays  the  program  was  canceled  in 
2004.  The  gap  in  products  was  filled  with  systems  from  another  manufacturer.  While 
many  of  the  ideas  and  some  of  the  technology  from  the  Eagle  program  were  carried 
forward  to  a  follow-on  program  based  on  the  “Rock”  microprocessor,  this  program  was 
also  delayed  and  canceled  in  2009. 


3.5.2.3  Thinking  Mac hines Vec  10 rUsiit Reject 

In  1990,  a  project  was  initiated  to  increase  the  floating-point  performance  of  the  CM-5 
supercomputer  by  a  factor  of  at  least  iox  to  be  deployed  in  volume  in  less  than  2  years. 
This  project  included  the  development  of  new  hardware,  OS  extensions,  run-time 
system  modifications,  math-library  development  and  new  application  code.  There  was 
great  clarity  on  the  value  of  the  project  for  the  CM-5  customers.  A  number  of  concepts 
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for  the  product  were  explored  and  discussed  across  the  company.  Being  a  start  up, 
there  were  a  limited  number  of  internal  stakeholders  all  of  whom  also  had  the  authority 
to  make  decisions  for  their  organizations.  Rather  than  creating  textural  descriptions  of 
the  evolving  concepts,  executable  models  were  created  which  provided  the  necessary 
representations  of  the  concept,  architecture  and  implementation  to  the  appropriate 
stakeholders  and  development  and  support  personnel  including  hardware,  software, 
application  and  service  support  staff.  Commitments  were  made  between  each 
organization  as  the  concepts  evolved  and  were  translated  into  the  architecture  and 
implementation.  The  new  result  was  that  the  system  was  delivered  within  two  years, 
performed  as  expected  and  had  a  very  long  life  in  the  field  (over  10  years  in  some 
installations). 


3.5.2.4  Rio  Grande  Studios 

Rio  Grande  Studios  is  a  fully  digital  studio  located  in  Albuquerque,  NM.  When  dealing 
with  investors  for  a  new  project,  they  create  a  10-12  minute  "movie"  of  the  project  in  7- 
10  days  with  2-3  people.  It  is  built  using  software  that  leverages  a  module  library  of 
items  that  can  be  put  into  motion.  Figures  9  and  10  are  examples  of  the  types  of 
capabilities  available  in  Storyboard  Lite.  Useable  objects  include  landscape,  people  and 
objects.  The  software  then  puts  the  scenario  into  motion.  Other  popular  tools  used 
include  Autodesk  Maya,  and  FrameForge  Pre-viz  Studios.  While  the  software  requires 
skilled  animators,  the  same  is  true  for  any  modeling  environment  today.  One  does  not 
sit  down  to  a  CAD  program  without  extensive  training. 


The  purpose  of  the  short  movie  is  to  develop  a  shared  vision  of  where  the  larger  project 
is  going.  The  stakeholders  of  this  shared  vision  include  the  director,  the  screenwriter, 
major  investors,  producer,  and  even  the  key  cast. 


3  http :  /  /  www.frameforge3d.  com  /  newsite  / 

4  http:  / /venicedna.com/ 
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Figure  10:  Software  for  Creating  Movies  from  Modules  -  Storyboard  Lite5 


3.5.2.5  Innovation  &  Design  Companies 

There  are  a  number  of  successful  innovation  and  design  organizations  such  as  IDEO, 
Frog  Design,  Applied  Minds  and  the  like  which  specialize  in  the  business  of  creating 
design  concepts  for  customers  in  a  number  of  diverse  fields.  While  each  of  these 
companies  have  distinctly  different  personalities,  each  of  them  rely  upon  getting  a  deep 
understanding  of  the  value  proposition  through  interaction  with  the  end  customers 
and/or  questioning  their  customer’s  understanding,  active  brainstorming  sessions  with 
multidisciplinary  experts  who  are  accustomed  to  working  together,  and  rapid 
prototyping  of  ideas.  This  approach  addresses  many  of  the  challenge  areas  for  CONOPS 
development. 

First  and  foremost,  these  firms  strive  to  understand  the  ultimate  value  proposition  from 
the  end  users  perspective.  Second,  they  translate  conceptual  decisions  into  easily 
perceivable  attribute  often  through  the  use  of  rapid  prototyping.  Next,  they  augment 
the  ability  to  form  shared  mental  models  amongst  people  with  diverse  capabilities  and 
backgrounds  by  forming  studios  or  teams  of  people  that  work  well  together  and  have 
formed  the  means  for  effective  communication.  Finally,  they  are  outside  firms  and  are 
largely  devoid  of  the  conflicts  of  interest  that  often  plague  organizations  -  they  are  only 
looking  for  the  solution  that  creates  the  most  value.  However,  this  independent  view 


5  http :  /  /  www.zebradevelopment.com/index.php 
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can  be  compromised  by  who  in  the  customer  organization  is  judging  the  success  of  the 
project.  At  the  end  of  the  day  though,  market  success  determines  the  success  of  the 
projects,  and  the  innovation  and  design  companies  with  the  most  successful  projects 
tend  to  flourish  and  those  that  don’t  often  fail. 


3.6  Identified C halleng es in  C reaung  CONOPS 

Several  common  challenges  can  be  derived  from  the  conducted  research  and  interviews. 
Despite  the  negative  impact  and  high  level  of  risk  incurred  by  not  having  a  consistent 
view  of  the  CONOPS  for  a  system  throughout  the  lifecycle,  it  is  difficult  to  create  an 
effective  one  due  to  a  number  of  important  challenges.  One  of  the  major  challenges  is 
that  the  development  of  a  CONOPS  often  involves  making  tradeoffs  in  a  large,  highly 
dimensional,  complex  trade  space  with  stakeholders  from  multiple  disciplines.  This  is  a 
difficult  task  for  the  following  reasons: 

1.  Value  Proposition:  There  is  generally  a  great  deal  of  uncertainty  in  the  value 
proposition  for  the  system  due  to  changing  market  and  competitive  conditions. 
Thus,  it  is  challenging  to  translate  system  attributes  into  realizable  value. 

2.  Translation  of  Concept  Decisions  into  System  Attributes:  Even  with  a  common 
understanding  of  the  value  proposition  space,  it  is  often  very  difficult  to  translate 
a  myriad  of  concept  decisions  into  the  impact  on  the  attributes  of  the  system  that 
are  pertinent  for  the  creation  of  value. 

3.  Shared  Mental  Model  Capability:  With  a  group  of  stakeholders  from  multiple 
disciplines,  it  may  be  very  difficult  to  create  consistent  shared  mental  models  or 
even  to  have  the  ability  to  determine  the  existence  of  inconsistencies  in  the 
mental  models  that  do  exist.  Unfortunately,  these  inconsistencies  are  often  found 
much  later  in  the  system  lifecycle. 

4.  Human  Dimension:  The  human  dimension  generally  manifests  itself  as  conflicts 
in  how  the  relative  costs  and  benefits  of  a  decision  are  distributed  to  the 
stakeholders.  This  is  an  area  where  reward  systems  and  organizational  structures 
are  extremely  influential.  Another  aspect  of  the  human  dimension  is  the  delaying 
effect.  This  may  manifest  itself  in  the  form  of  engaging  in  delaying  tactics  as  a 
method  to  avoid  conflict,  or  to  stall  a  decision  because  the  key  stakeholders 
and/or  decision  makers  have  differing  opinions. 

5.  Process  Nonlinearity:  The  prevalent  practice  of  developing  CONOPS  has 
attempted  to  apply  a  linear  approach  to  an  inherently  nonlinear  process. 
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4  Addressing theCONOPSChallenges 


In  this  section,  we  present  an  agile  CONOPS  process  and  a  graphical  CONOPS  system. 
Together,  these  two  components  address  the  aforementioned  challenges  in  a  number  of 
ways  as  we  demonstrate  in  the  following: 

1.  Value  Proposition:  Our  CONOPS  process  allows  the  CONOPS  team  and 
stakeholders  to  revisit  the  original  concept,  conduct  risk  analyses,  evaluate  trade¬ 
offs,  etc.  at  multiple  points  throughout  the  process.  At  each  of  these  points, 
current  market  and  competitive  conditions  can  be  incorporated  to  ensure  that  the 
value  proposition  is  realized. 

2.  Translation  of  Concept  Decisions  into  System  Attributes:  Our  graphical  CONOPS 
system  will  facilitate  translating  concept  decision  into  system  attributes.  The 
graphical  interface  we  envision  will  provide  the  CONOPS  team  and  stakeholders 
with  the  ability  to  visualize  both  the  concept  and  the  system  attributes,  and  more 
importantly,  compare  them. 

3.  Shared  Mental  Model  Capability:  The  capability  to  create  shared  mental  models 
is  enhanced  through  both  our  CONOPS  process  and  graphical  CONOPS  system. 
The  process  is  designed  to  help  all  participants  attain  shared  mental  models 
about  the  desired  future  state  during  the  Conceptual  Phase,  by  incorporating  the 
views  of  all  stakeholders  throughout  the  process,  and  by  negotiating  to  resolve 
conflicts  in  real  time.  The  system  will  include  visualization  tools  (e.g.,  concept 
maps,  Systemigrams)  that  facilitate  shared  mental  model  development. 

4.  Human  Dimension:  By  incorporating  stakeholders  into  the  CONOPS  process, 
conflict  and  delay  tactics  can  be  minimized.  For  instance,  CONOPS  documents 
will  not  wait  on  stakeholders’  desks  until  they  have  the  time  to  evaluate  them 
since  the  stakeholders  will  be  actively  participating  in  real  time.  Through  this 
active  participation,  stakeholders  may  be  compelled  to  reach  consensus  as  they 
feel  ownership  in  the  process. 

5.  Process  Nonlinearity:  The  proposed  CONOPS  process  incorporates  multiple 
feedback  and  feed-forward  loops  that  allow  the  participants  to  reconsider  their 
current  activities  based  on  earlier  decisions  and/or  new  information. 
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4.1  Agile  CO  NO  PS  Process 

We  propose  the  following  three-stage  process  for  agile  CONOPS  development: 

Stage  1)  Conceptual  Phase 

Stage  2)  Specification  Phase 

Stage  3)  Design  and  Implementation  Phase 


While  in  theory  these  stages  are  the  standard  phases  used  in  current  CONOPS 
processes,  our  process  is  reliant  upon  stakeholder  input  throughout  the  process  (vs.  at 
the  end),  formally  emphasizes  the  conceptual  phase  (vs.  embedding  it  as  a  component  of 
the  specification  phase),  and  incorporates  feedback  and  feed-forward  loops  to  ensure 
that  the  original  intention  is  not  lost  (vs.  a  primarily  linear  approach  that  may  diverge 
significantly  as  the  process  evolves).  In  addition  to  these  improvements,  we  have 
designed  a  robust  process  that  is  applicable  to  a  wi de-array  of  CONOPS,  across 
disciplines,  and  throughout  a  product/process  life-cycle. 


4.1.1  Siage  1  -  Conceptual  Phase 

Figure  11  shows  the  conceptual  phase  and  related  tools  and  methods.  The  CONOPS 
process  essentially  begins  with  a  perceived  need  that  is  expressed  either  through  formal 
channels  (top-down)  or  informal  channels  (bottom-up).  It  ultimately  results  in  a 
decision  to  proceed  with  a  CONOPS  for  a  new  system,  for  modifications/upgrades  of  an 
existing  system  or  for  establishing  operational  strategy.  The  core  team  can  then  use 
stakeholder  participation  heuristics  and  frameworks  such  as  the  PLP  (Participation 
Level  Points)  heuristic  and  the  SPK  (Stake  Power  Knowledge)  framework  introduced  in 
Appendices  A  and  B  to  identify  the  optimal  level  of  participation  and  the  relevant 
stakeholders  for  collaborative  development  of  the  CONOPS.  The  needs,  interests  and 
perspectives  of  the  stakeholders  can  then  be  mapped  using  initial  surveys  and 
interviews.  In  joint  sessions  with  all  stakeholders  the  problem  definition  is  refined  and 
the  desired  state  future  state  of  the  system  is  mapped.  Once  this  iterative  process  has 
resulted  in  a  shared  mental  model,  the  desired  future  state  is  refined  at  a  conceptual 
level  and  the  group  can  proceed  to  the  specification  phase.  The  conceptual  phase  is  an 
important  phase  as  many  organizations  rush  into  the  specification  phase  without  a  clear 
agreement  on  the  situational  analysis.  Table  8  also  shows  the  set  of  tools  and  methods 
that  can  be  leveraged  at  this  stage  to  increase  the  effectiveness  of  the  process. 
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Mandate  a 


Form  Core 
CONOPS 
Facilitation 


Figure  11:  Stage  1  Conceptual  Phase 
lable  8:  Stage  1  Conceptual  Phase  Process  Steps,  tools,  and  Methods 


CONOPS  Process  Steps 

Tools 

Methods 

Mandate  CONOPS 

Web-based  Templates 

Objective  Statement 

Form  CONOPS  Core  Team 

SPK  Framework 

Identify  Stakeholders 

Web-based  Heuristic 

SPK  Framework 

Elicit  Stakeholder  Inputs 

Surveys,  Interviews 

Discourse  Integration, 
Contextual  Analysis,  Data 
Analysis 

Conceptual  Situational 
Analysis  (Define 
problem/define  desired 
state/identify  gap) 

Visual  Tools 
(Modular/lego, 
storyboarding,  causal 
loop  diagrams  etc.) 

Brainstorming,  Consensus 
seeking,  Shared  Mental 

Models,  traditional  research 
methods  etc. 

4.1.2  Siage 2  -  THe Specification  Phase 

Taking  the  output  from  Stage  l  and  identifying  any  new  stakeholder  groups  that  need  to 
be  involved,  stakeholder  requirements  are  mapped  and  a  trade-off  analysis  is  conducted 
to  assess  the  feasibility  of  the  requirements  serving  as  a  basis  for  negotiations  between 
the  user  community,  the  developers  and  the  decision-makers  on  desired  future  specs 
and  their  prioritization.  Comparing  the  desired  future  specs  with  existing 
specs/capabilities  allows  the  participants  to  identify  the  gaps,  gather  technical  data, 
conduct  risk  analyses  on  features  of  the  desired  future  state  and  finalize  the  detailed 

Contract  Number:  H98230-08-D-0171  DO  001,10  002  RT3  CONOPS 

Report  No.  SERC- 2009- TR- 003 
October  30,  2009 

UNCLASSIFIED 

40 


UNCLASSIFIED 


specs/requirements  of  the  desired  future  state  of  the  system.  A  variety  of  methods  such 
as  discourse  integration,  contextual  analysis,  data  analysis,  utility  theory,  multi¬ 
attribute  Trade-off  analysis,  Consensus  Seeking  Negotiations,  Group  Brainstorming, 
Consensus  seeking,  Shared  Mental  Models,  Traditional  Research  methods,  Risk 
Analysis  etc.  can  be  leveraged  at  this  stage  to  get  to  the  final  spec/requirements  output 
for  the  desired  future  system.  Figure  12  and  Table  9  show  this  iterative  process  and  the 
tools  and  methods  that  can  be  used  at  each  step  of  the  process. 


Input:  Desired 
Future  State 


Elicit/map 

Stakeholder 

requirements 


Define  Current 
State/  Capabilities 


Evaluate 

.Tradeoffs 

Negotiate  on 
Capabilities/ 
Specs 


Gather  Technical 
Information/  Conduct 
Risk  Analysis 


Define  Desired 
Future 

Capabilities/Specs 


Identify 

State/Capabilities  Gap 


Output:  Desired 
Future 

Capabilites/Specs 


Figure  12:  Stage  2  Specification  Phase 
table  9:  Stage  2  Specification  Phase  Process  Steps,  Tools,  and  Methods 


CONOPS  Process  Steps 

Tools 

Methods 

Elicit/Map  Stakeholder  Technical 
Requirements 

Surveys,  Interviews, 
Face-to-face  discussions 

Discourse  Integration,  Contextual 
Analysis,  Data  Analysis,  Utility 

Theory 

Evaluate  Tradeoffs  (Capab/Specs) 

Multi-attribute  Decision- 
support  Tools 

Multi-attribute  Trade-off  analysis 

Negotiate  on  Capabilities/Specs 

Model-based  Negotiation 
Tools 

Consensus  Seeking  Negotiations 

Technical  Situational  Analysis 
(Define  current  sit/define  desired 
cap-specs/identify  gap/risk  analysis 
and  information  gathering) 

Visual  Tools 
(Modular/Lego, 
storyboarding,  simulation 
tools  etc.) 

Brainstorming,  Consensus  seeking, 
Shared  Mental  Models,  traditional 
research  methods,  risk  analysis  etc. 
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4.1.3  Siage  3  -  Design  and  Implemeniahon  Phase 

In  the  final  stage  the  inputs  from  the  specification  phase  serve  as  a  basis  to  identify  the 
detailed  system  components  and  interfaces  needed  to  achieve  the  desired  capabilities 
and  identify  the  exact  team  that  will  manage  and  implement  the 
development/deployment/usage  of  the  system.  Using  tradeoff  analysis  to  identify 
priorities  in  the  design  and  implementation  of  the  desired  future  system,  the  overall 
system  architecture  and  a  management  and  implementation  plan  can  be  negotiated 
among  the  stakeholders.  This  again  is  an  iterative  process  and  the  cycles  end  when 
stakeholders  converge  on  an  agreement  for  the  two  outputs.  Figure  13  and  Table  10 
show  the  process  steps  for  this  stage  and  the  relevant  tools  and  methods  that  can  be 
leveraged. 


Capabilites/ Specs 

Identify  Potential  System 
Components  and 
Interfaces 

V 

Evaluate 
Tradeoffs 


Identify  Stakeholder 
Re  s  our  c  es/Mandates 


Output:  System 
Architecture 


Negotiate  on 
Implementation  and 
Management  Plan 

I 


Figure  13:  Stage  3  Design  and  Implementation  Phase 
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Table  10:  Stage  3  Design  and  Implementation  Phase  Process  Steps,  Tools,  and  Methods 


CONOPS  Process  Steps 

Tools 

Methods 

Identity  Stakeholders 
Resources /Mandates 

Surveys,  Interviews,  Face- 
to-face  discussions 

Identify  System  Components 

System-specific  and 
Technical  Models 

Brainstorming,  Consensus 
seeking,  Shared  Mental 

Models,  traditional  research 
methods,  risk  analysis  etc. 

Evaluate  Tradeoffs 
(Components, 

Implementation  and 
Management  Plan) 

Multi-attribute  Decision- 
support  Tools 

Multi-attribute  Trade-off 
analysis 

Negotiate  Implementation 
and  Management  Plan 

Model-based  Negotiation 
Tools,  Project 

Management  Software 

Consensus  Seeking 
Negotiations,  Project 
Management 

4.2  Graphical  CO  NO  PS  System 

While  there  are  a  number  of  interesting  and  exciting  technologies  available  today  to 
assist  in  developing  a  graphical  CONOPS,  no  tools  or  integrated  systems  exist  that  are 
specifically  created  for  this  expressed  purpose.  We  propose  developing  the  Graphical 
CONOPS  System  shown  in  Figure  14.  A  number  of  technologies  are  available  to  create 
such  a  graphical  CONOPS  tool,  and  they  are  discussed  in  the  following  sections. 


Graphical  interface  to  allow 
interactive,  collaborative  creation 
of  modular  CONOPS  model 


Portfolio  of  Canned  Scenarios 


DOTMLPFP  Simulator 


Cause  &  Effect  Analysis 


Customizable  Presentation 
of  Results 


Figure  14:  Representation  of  Graphical  CONOPS  System 
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4.2. 1USER INTERFAC  E 

A  Graphical  Concepts  of  Operation  can  be  created  by  dragging  and  dropping  CONOPS 
primitives  onto  a  common  visualization  palette  and  defining  the  interconnections 
between  the  primitives  via  a  user  interface.  Along  with  a  set  of  standard  CONOPS 
primitives,  the  users  can  also  be  provided  with  a  set  of  easily  customizable  primitives, 
CONOPS  patterns,  and  various  test  scenarios.  CONOPS  developments  would  both 
leverage  previous  work  and  create  new  models,  patterns  and  test  scenarios  that  may  be 
used  for  future  CONOPS  development,  both  updates  and  for  new  systems. 

The  user  interface  could  involve  a  small  group  working  on  a  single  Surface  Computing 
(MS)  table  touch  table  in  which  everyone  shares  the  same  view  (perhaps  with  a  small 
window  for  a  personalized  view).  Alternately,  each  person  could  use  a  single  mouse  (e.g., 
MS  Multipoint)  with  a  large  monitor,  or  could  work  with  the  own  monitor  (with  a 
personalized  view)  and  keyboard  with  networked  PCs  with  a  shared  large  monitor. 
Multiple  sites  can  be  networked  together  (albeit  with  some  restrictions  to  handle 
potential  latency  issues)  to  provide  for  a  distributed  development  experience.  The 
important  aspect  of  this  is  that  each  participant  can  freely  and  independently  make 
changes  to  a  single  CONOPS  model,  while  sharing  a  common  view  and  perhaps  having  a 
customized  local  view  as  well. 

The  primitives  can  be  represented  by  smart  blocks,  such  as  Siftables,  or  simple  graphical 
objects.  The  different  strengths  associated  with  each  of  the  interconnections  between 
CONOPS  primitives  can  be  represented  using  interaction  modeling  technologies,  such 
as  those  used  for  physical  chemistry  to  create  a  model  which  effectively  provides  a 
minimal  energy  state.  The  primitives  and  interconnections  are  coupled  with  simulation 
models  and  analysis  procedures.  Creation  of  a  Graphical  CONOPS  model  via  the  user 
interface  leads  to  the  creation  of  the  integrated  simulation  and  analytical  model 
representing  the  system  under  consideration. 


4.2.2  3D/  2D  VIEWING  &  USER  IO 

Logo  was  at  one  time  the  tool  of  choice  for  elementary  education  to  introduce  students 
to  the  concept  of  computer  programming  using  a  simple  language  to  cause  a  tethered 
“turtle”  to  move  around  a  room.  Over  the  years,  the  language  has  matured  to  become  a 
serious  agent-based  programming  language.  StarLogo  TNG6  is  such  a  tool.  An  agent- 
based  program  is  constructed  by  assembling  lego-style  primitives  into  an  executable 
computer  simulation  (Figure  15). 


6  http:  / /education.mit.edu/drupal/starlogo-tng 
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Another  potential  technology  for  viewing  CONOPS  is  the  Microsoft's  Surface  Computing 
touch  table  (or  a  number  of  other  similar  implementations)  mentioned  above.  This 
would  allow  a  number  of  people  to  simultaneously  interact  with  graphical  building 
blocks  to  construct  a  CONOPS.  In  addition,  this  could  be  extended  to  multiple  sites  and 
distributed  collaboration  with  the  networking  of  a  number  of  tables. 


Figure  15:  SarLogo  TNG  Modeling  Environment 

Templates,  or  patterns,  could  be  used  to  quickly  create  new  combinations  or  variants  of 
existing  CONOPS.  Tapping  the  blocks  could  be  used  to  dive  deeper  into  the  hierarchy, 
squeezing  could  allow  you  to  pop  up  in  the  hierarchy. 

A  less  expensive  technology  to  that  of  Surface  Computing,  is  the  Multipoint  software 
development  kit  (SDK)  which  can  be  downloaded  to  assist  in  the  development  process. 
Using  mice  as  the  primary  interface  will  certainly  enforce  a  simple  interface  with  the 
program.  One  of  the  challenges,  of  course,  is  avoiding  conflicts  on  the  screen  (the  SDK 
allows  the  support  of  rules,  privileges,  hierarchy,  etc.  for  collision  situations),  and 
keeping  it  a  constructive  operation.  A  collaborative  etiquette  would  need  to  be 
developed.  Another  potential  technology  is  HubneD  which  is  integrated  with  NetLogo. 


7  http://ccl.northwestern.edu/netIogo/hubnet.html 
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4.2.3  PRESENTATION  OF  INFORMAHON 

This  system  can  provide  a  number  of  different  views,  both  2D  and  3D.  The  2D  views  can 
be  of  an  entire  system  or  of  a  particular  layer  in  a  system  hierarchy.  Moving  to  a  3D 
view,  various  layers  in  the  hierarchy  can  be  presented.  What  is  represented  in  a 
particular  layer  can  be  selected  by  each  user  which  enables  them  to  view  the  system 
from  a  number  of  different  vantage  points.  With  multiple  screens,  a  number  of 
participants  can  each  interact  with,  build  and  modify  the  system  simultaneously,  each 
from  their  own  perspective.  A  global  view  could  be  presented  to  all  on  the  large  screen 
on  a  wall  while  each  participant  interacts  on  their  own  PC/laptop.  An  example  of  such  a 
flexible  digital  dashboard  can  be  seen  in  Figure  16,  which  shows  the  Composable  Digital 
Dashboard  developed  at  Texas  A&M  University  for  situational  common  operating 
picture  in  emergency  response  situations.  Another  example  of  a  visualization  platform  is 
the  EWall  system,  developed  by  MIT  to  assist  in  group  cognitive  processes. 


Figure  16:  Composable  Digital  Dashboaid  for  Common  Operating  Picture  of  Emeigency 

Response  Scenario 

Finally,  each  user  can  have  personalized  information  relevant  for  the  stakeholders  that 
he/she  represents  on  his/her  view.  This  information  can  be  customized  to  provide  users 
with  what  they  need  to  see,  while  the  information  pertaining  to  the  global  good  can  be 
viewed  on  a  global  screen. 
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It  should  be  noted  that  both  the  CONOPS  structure  and  the  simulation  outcomes  can  be 
personalized  for  each  of  the  viewers.  It  might  be  interesting  for  the  various  stakeholders 
to  swap  screens  and  see  how  the  world  looks  from  the  others  perspectives.  Perhaps  this 
could  be  a  mandatory  practice  during  the  CONOPS  development  process. 


4.2.4  Information  Mapping 

Information  mapping  is  a  technique  for  analyzing,  organizing,  and  presenting 
information  that  is  dependent  upon  the  needs  and  purposes  for  which  the  information  is 
being  collected.  It  is  routinely  used  for  activities  including  information  visualization, 
knowledge  management,  graphic  design,  etc.  The  results  are  considered  easy  to 
comprehend,  use,  and  recall8. 


42.4.1  Systemigrams 

Systemigram  is  a  portmanteau  word  taken  from  systemic  and  diagram.  The 
systemigram  is  an  approach  to  graphically  tell  a  story,  and  is  very  useful  in  creating  a 
shared  understanding  of  the  problem  at  hand.  It  is  a  structured  translation  of  the  words 
and  meanings  that  is  reminiscent  of  sentence  diagramming  that  used  to  be  taught  in 
elementary  school,  but  is  much  more  powerful  and  easily  understood. 

The  primary  sentence  (mainstay)  which  supports  the  purpose  of  the  system  will  read 
from  top  left  to  bottom  right.  Ideally  there  should  be  15-25  nodes  (less  can  make  for  a 
trivial  system  description;  more  can  create  clutter  and  illegibility).  Nodes  must  contain 
noun  phrases,  and  links  should  contain  verb  phrases  (to  reduce  trivial  links).  There 
should  be  no  repetition  of  nodes,  and  no  cross-over  of  links. 

When  creating  a  systemigram,  the  author  works  with  subject  matter  experts  to  dialogue 
the  CONOPS.  It  is  important  for  the  author  to  remember  that  the  model  (systemigram) 
is  really  ‘theirs’  (belongs  to  the  client).  All  the  author  does  is  to  present  a  fresh 
perspective  on  their  system  descriptions  with  hopefully  some  added  value. 

Figure  17  is  a  representation  of  an  intelligence  community,  developed  by  Dr.  John 
Boardman. 


8  Information  mapping.  (2009,  June  23).  In  Wikipedia,  The  Free  Encyclopedia.  Retrieved  18:30,  October 
24,  2009,  from  http:  / /en.wikipedia.org/w/index.php?title=Information  m a p n i n g & o  1  d i d = 2 9 8 o 6 r-i 4 8 o 
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Figure  17:  Intelligence  Community  Systemigram9 


4.2.4.2  MindMaps 

A  mind  map  is  a  picture  which  represents  ideas,  captured  as  a  combination  of  words  and 
cartoons  around  a  central  thought  or  concept.  It  was  developed  by  Tony  Buzan  in  the 
‘70’s,  and  he  coined  the  term  “Mental  Literacy”.  Figure  18  is  a  mind  map  for  creating 
mind  maps. 

Mind  maps  are  used  to  help  visualize  thoughts  or  notes  as  they  are  developed,  providing 
multiple  senses  stimulus  to  improve  understanding  and  recall.  Practitioners  are 
encouraged  to  use  multiple  colors,  graphics,  etc.  For  the  more  computer  minded,  there 
are  tools  such  a  Mind  Manager9 10  and  Personal  Brain.  Figure  19  is  a  project  scope 
definition  example  from  Mind  Manager. 


9  http :  / /www.boardmansauser  .com  /  thoughts  /  systemigrams  .html 

10  http://www.mindjet.com/ 
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Figure  18:  Mind  Map  on  howto  Mind  Map11 


11  Michael  J.  Gelb,  How  to  Think  Like  Leonardo  Da  Vinci,  p  174,183. 
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Figure  19:  Example  Mind  Map  from  Software 


4.2.43  ConceptMapsandSidryboarding 

Concept  maps  allow  for  the  graphical  representation  of  concepts  using  nodes  to 
represent  information  pieces,  and  arcs  to  represent  the  linkages  between  the  nodes  of 
information.  They  are  often  used  to  develop  and  document  mental  models.  Much  of  the 
research  to  date  has  focused  on  structural  similarity  when  the  nodes  are  given  to  the 
study  participants.  Concept  maps  may  be  a  valuable  tool  for  CONOPS  if  stakeholders  are 
allowed  to  provide  their  own  nodes  in  at  least  3  ways.  Specifically,  they  will:  (1)  help  the 
stakeholders  develop  a  common  language,  (2)  facilitate  making  linkages  among  key 
concepts,  and  (3)  enable  the  establishment  of  shared  mental  models. 

Concept  maps  could  be  used  to  help  create,  communicate,  and  refine  CONOPS 
information.  The  tool  helps  to  represent  the  information  visually,  integrate  it  with 
storyboard  sketches,  refine  it  through  interaction  with  other  stakeholders,  and  finally 
export  the  information  in  a  form  directly  usable  by  developers.  Currently  it  is  possible  to 
create  two  dimensional  and  three  dimensional  animated  storyboards  using  Photoshop, 
AfterEffects,  3D  Studio  Max  or  Maya  that  animate  realistic,  technically  and  physically 
accurate,  modern-day  military  scenarios  for  CONOPS  presentations. 


4. 2.4.4  Tree  Maps 

Tree  maps  display  hierarchical  (tree-structured)  data  as  a  set  of  nested  rectangles.  Each 
branch  of  the  tree  is  given  a  rectangle,  which  is  then  tiled  with  smaller  rectangles 
representing  sub-branches.  A  leaf  node's  rectangle  has  an  area  proportional  to  a 
specified  dimension  on  the  data.  (In  the  illustration,  this  is  proportional  to  a  waiting 
time).  Often  the  leaf  nodes  are  colored  to  show  a  separate  dimension  of  the  data12. 

When  the  color  and  size  dimensions  are  correlated  in  some  way  with  the  tree  structure, 
one  can  often  easily  see  patterns  that  would  be  difficult  to  spot  in  other  ways.  A  second 
advantage  of  treemaps  is  that,  by  construction,  they  make  efficient  use  of  space.  As  a 
result,  they  can  legibly  display  thousands  of  items  on  the  screen  simultaneously. 


43.4.5  Map  of  the  Market 

The  Map  of  the  Market  is  an  example  of  a  treemap.  The  map  is  a  graphical 
representation  which  tracks  the  stock  market  action  for  over  500  US  based  stocks. 
Developed  by  Martin  M.  Wattenberg,  a  scientist  and  artist  known  for  his  work  with  data 
visualization,  it  is  based  on  a  modified  treemap  algorithm.  Each  rectangle  on  the  map 


12  http://en.wikipedia.org/wiki/Treemap 
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represents  an  individual  stock.  If  the  color  is  green,  the  stock  is  up.  The  larger  the  box, 
the  larger  the  market  capitalization  that  stock  represents.  This  becomes  a  quick  and  easy 
way  to  see  what  sectors  and  stocks  are  causing  the  stock  market  to  move.  A  snapshot  of 
this  graphical  representation  can  be  found  in  Figure  20. 


Map  of  the  Market 

Launch  Map  in  Separate  Window  Q? 
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•  AmEx  03  Profit  Down  21%.  Users  Cut  Spending 

•  Stock  Screen:  3  Stocks  Trading  Near  Asset  Value 
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Figure  20:  Map  of  the  Market 


4.2.5  C  O  NO  PS  Smuiahon  Engine 

The  CONOPS  simulation  engine  will  have  the  ability  to  run  forward  and  backwards  in 
time,  with  specified  amounts  of  accuracy,  as  well  as  a  given  amount  of  predetermined 
time-steps  (generally  specified  in  terms  of  amount  of  storage  allocation  and  fidelity).  It 
will  also  be  possible  to  set  trap  conditions  to  stop  execution  and  trace  back  on  the  events 
that  caused  the  trap  condition  to  occur.  This  tracing  mode  will  be  useful  in  determining 
event  causality. 

The  simulation  engine  will  normally  run  in  interpreted  mode,  such  that  the  simulation 
can  be  stopped,  changes  can  be  made  (within  a  finite  range)  and  the  simulation  can  be 
started  again  without  a  lengthy  recompilation.  One  mode  of  operation  is  to  run  a  set  of 
test  scenarios  and  upon  failure,  trace  back  to  the  root  cause,  make  the  necessary 
corrections  and  run  from  that  point  forward  to  determine  if  the  problem  is  fixed  or  it 
causes  other  failures.  The  development  of  test  scenarios  can  be  likened  to  Test  Driven 
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Development  (TDD)  or  the  development  of  requirements.  As  the  team  gains  experience 
with  the  CONOPS,  the  test  scenarios  can  be  updated  and  enhanced  to  better  illustrate 
the  desired  behavior. 

Another  interesting  aspect  of  the  simulation  coupled  CONOPS  would  be  to  create  a 
retrospective  study  of  how  the  system  designers  reacted  after  seeing  the  performance  of 
their  proposed  configuration  (it  will  help  the  cognitive  scientists  gather  first  hand  data 
on  their  preferences,  utilities,  etc).  This  might  allow  one  to  see  blind  spots  in  group 
behavior  in  which  a  repetitive  set  of  responses  are  made  without  other  productive 
approaches.  The  system  would  have  access  to  all  of  the  information  input  into  the 
system,  and  audio/video  information  could  be  recorded  as  well.  The  system  could  very 
well  be  used  as  a  trainer  for  personal  and  team  development. 


42.5.1  Simulation  Engines 

It  is  believed  that  simulators  can  be  broken  into  two  distinct  classes  -  Immersive 
Experience  and  Simulators. 

An  Immersive  Experience  environment  can  be  thought  of  as  one  which  provides  the 
user  (game  player)  with  the  ability  to  have  an  immersive  experience  that  mimics  real 
life.  These  tend  to  be  physics-based  systems  that  simulate  a  human  operating  a  machine 
(flight  simulator,  tank  battle,  etc.),  or  human  avatars.  These  can  be  systems  with  a  single 
player,  one  with  a  single  player  and  computer  controlled  other  agents,  or  multiple 
agents.  Second  life  and  the  many  multi-player  gaming  environments  are  examples  of 
these. 

The  behaviors  of  the  agents  tend  to  be  simplified  and  the  effort  is  placed  in  creating  a 
realistic  environment  in  which  the  player  can  engage.  In  the  second  type,  the  effort  is  in 
creating  an  accurate  simulation  of  complex  processes  and  the  realism  of  the 
environment  is  secondary. 

Immersive  environments  tend  to  be  very  impressive.  They  can  be  very  high  definition 
and  photorealistic,  but  may  require  significant  time,  effort  and  cost.  Proving  that  one 
can  create  an  interactive  environment  which  models  sufficiently  complex  systems 
quickly  and  to  the  required  level  of  fidelity  is  likely  to  be  where  the  challenge  lies  in 
these  systems. 

Simulators  are  not  intended  to  provide  a  real-life  experience  for  a  player,  but  rather 
provide  a  simulation  of  how  a  component  or  system  might  behave.  This  has  the 
advantage  that  rate  of  time  change  is  a  variable  that  can  be  modified  dependent  on  the 
desired  results.  Finite-element  analysis,  weather  modeling,  etc.  are  examples  of  these. 
Other  examples  are  system  simulators  which  include  both  hardware  and  software 
subsystems  and  components. 
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For  graphical  CONOPS,  it  is  likely  that  the  simulator  type  of  visualization  is  most 
appropriate,  with  the  focus  on  ease  of  changing  the  behaviors  and  interactions  between 
the  agents.  The  AADL  language  appears  to  be  aimed  at  this  target  in  the  modeling  of 
HW/SW  systems.  Little  Jil  from  UMass  is  targeted  at  doing  this  for  the  modeling  of 
processes.  Other  languages,  such  as  SysML,  UML,  and  the  like  may  also  be  appropriate 
for  graphical  CONOPS. 


4.3  S^uent  Futures  of  Our  Answer  to  the  CO  NO  PS  Challenges 

The  CONOPS  Process  and  Graphical  Operation  we  have  presented  address  the 
challenges  we  identified  through  our  assessment  of  CONOPS  documents  and  interviews 
with  practitioners  familiar  with  the  CONOPS  process.  They  have  several  salient  features 
that  include: 

1.  Involving  relevant  stakeholders  in  all  phases  of  the  CONOPS  development. 

2.  Developing  a  system  that  facilitates  easy  visualization  and  agile  modifications  by 
large  numbers  of  stakeholders  with  varying  roles. 

3.  Assisting  shared  mental  model  development  throughout  the  three  process  stages 
by  leveraging  an  integrated  toolset  that  enables  stakeholder  participation. 


4.3.1  Stakeholder  Involvement 

Agile  CONOPS  development  requires  active  stakeholder  participation  from  the 
beginning,  not  just  after  the  fact.  While  this  notion  is  commonly  understood,  it  is  often 
overlooked  in  practice.  In  much  of  the  research  we  conducted,  stakeholders  were 
involved  after  the  preliminary  set  of  CONOPS  documents  were  completed  rather  than 
during  the  process  as  we  propose.  If  the  right  stakeholders  are  chosen  and  incorporated 
into  the  process,  the  advantages  are: 

•  Users,  developers  and  other  stakeholders  can  discuss  the  conceptual  need  for  a 
system,  negotiate  requirements,  and  coordinate  design,  development  and 
implementation  in  an  interactive  manner. 

•  Stakeholders  can  help  the  CONOPS  team  decide  which  steps  can  be  skipped  in 
the  CONOPS  process  without  compromising  quality. 

•  The  CONOPS  can  be  developed  much  faster  in  a  few  joint  sessions  conducted  in 
person  or  through  virtual  meetings. 

•  The  risks  with  systems  implementation  and  costs  can  be  better  estimated  having 
most  of  the  relevant  stakeholders  at  the  table. 

•  A  shared  mental  model  emerges  during  such  meetings  that  can  help  the  success 
of  the  system  during  development  and  implementation. 
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•  Feelings  of  ownership  evolve  within  a  wider  community  across  the 
organization(s). 


4.3.2  AG  lUTY  AND  VlSUAUZAHON 

While  text  based  CONOPS  have  been  traditionally  used,  they  have  several  limitations 
when  used  in  multi-user,  distributed  settings.  For  example,  editing  text  documents 
collectively  and  interactively  is  difficult.  Furthermore,  text  based  CONOPS  cannot  be 
easily  tailored  to  be  of  use  to  different  roles  within  the  system.  A  graphical  CONOPS  can 
have  the  following  advantages: 

•  Ease  of  access  to  various  sections  by  different  stakeholders 

•  Simultaneous  editing  possibility  for  various  stakeholders  in  interactive  sessions 

•  Ease  of  reaching  a  shared  mental  model  among  stakeholders 

•  Modular,  easily  modifiable  format 

•  Can  be  mined  later  to  create  standard  text  documents 

•  More  intuitive  for  later  referencing 


4.3.3  Shared  Menial  Models 

Shared  mental  models  play  a  critical  role  in  all  stages  of  the  CONOPS  process.  Shared 
mental  models,  however,  are  not  generic.  Rather,  they  contain  specific  content  that  is 
dependent  upon  the  purpose  of  the  particular  phase,  or  even  of  the  particular  step  in  the 
phase.  For  example,  much  of  the  Conceptual  phase  requires  developing  an  action- 
oriented  plan  where  task-relevant  content  must  be  shared  in  order  to  ensure  that  all 
perspectives  are  accounted  for  in  the  description  of  the  desired  future  state. 
Alternatively,  the  various  negotiation  steps  within  the  Specification  and  Design  and 
Implementation  phases  maybe  considered  cognitive  conflict  tasks  (McGrath,  1984).  As 
such,  shared  mental  models  addressing  each  team  member’s/stakeholder’s  point  of  view 
may  be  necessary  to  devise  a  feasible  plan  suitable  to  all  parties.  For  a  more 
comprehensive  description  of  shared  mental  models,  see  Appendix  C.  Regardless  of  the 
content,  the  CONOPS  process  we  developed  will  help  expedite  the  creation  of  shared 
mental  models.  Quickly  creating  shared  mental  models  will  benefit  the  CONOPS  process 
in  several  ways  including: 

•  Establishing  consistent  perspectives  that  will  govern  activities  throughout  the 
various  stages 

•  Reducing  the  overall  time  to  develop  CONOPS 

•  Minimizing  problems  due  to  misunderstandings  that  often  occur  toward  the  end 
of  CONOPS 

•  Ensuring  that  the  customers  get  the  product/process  that  meets  their 
expectations 
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5  Recommendations  For  Rjrjre  Research 


The  following  are  a  set  of  the  greatest  challenges  to  the  implementation  of  the  graphical 
CONOPS  system.  Each  of  these  areas  represents  primary  areas  for  future  research.  The 
sections  below  describe  some  of  the  work  that  can  be  done  in  a  modular  approach. 


5.1  Defining  ihe  Building  Blocks  fora  Graphical  CO  NO  PS 

To  rapidly  build  a  graphical  CONOPS  using  some  form  of  automated  tools,  a  collection 
of  primitives  and  building  blocks,  and  a  composition  language  are  critical.  An  open 
question  is  how  many  primitive  types  are  required  to  model  an  appropriately  detailed 
CONOPS  and  at  what  level  of  complexity  should  they  be  constructed  for  optimal  use.  If 
a  reasonably  small  number  of  primitives  are  created,  will  there  be  an  unacceptable 
amount  of  work  necessary  to  build  them  up  to  represent  useable  concepts?  Can  the 
interfaces  be  standardized  sufficiently  to  permit  the  use  of  on-the-fly  use?  The  CONOPS 
analysis  indicated  3  different  classes  of  CONOPS:  1)  Development  of  a  New  System,  2) 
Modification/  Upgrade/  Change  to  Existing  System/Product,  and  3)  Operational 
Strategy  (which  may  also  include  end  of  life  activities).  However,  the  interviews 
indicated  a  different  set  of  CONOPS  classes:  Products,  Continual  Services,  and 
Customized  Responses.  Will  each  of  these  classes  of  CONOPS  require  different 
primitives  or  are  they  likely  to  be  similar?  Further  research  should  be  conducted  to 
address  this  question. 

The  tasking  in  sections  5.1.1  and  5.1.2  could  be  done  iteratively,  decomposing  one  class 
of  CONOPS  first,  and  understanding  the  necessary  primitives,  semantics  and  building 
blocks  before  moving  to  another  class  of  CONOPS  -  each  iteration  building  on  the 
previous  iteration.  The  investigation  could  also  be  parsed  by  CONOPS  domains  if 
desired. 


5.1.1  CONOPS  Definition,  Decomposition,  &  Primitive 
Development 

The  first  step  of  creating  a  graphical  CONOPS  toolbox  would  be  to  identity  and  create  a 
base  set  of  core  primitives  which  would  then  become  the  building  blocks  for  more 
complex  functions  and  actions.  To  do  this,  a  consistent  taxonomy  of  CONOPS  terms  and 
definitions  should  be  created.  This  collection  of  terms  and  definitions  will  then  be  used 
to  define  core  primitives  and  objects,  their  attributes  and  actions,  in  a  generic  and 
reusable  form  for  a  tool  based  approach.  This  would  entail  the  decomposition  of  a 
representation  CONOPS  and  the  development  of  a  set  of  primitives  necessary  to  most 
efficiently  and  effectively  model  it.  Examples  of  initial  primitives  may  include  move, 
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communicate,  engage,  observe,  decide,  change,  etc.  Core  objects  will  include  items  such 
as  vehicle,  person,  organization,  etc.  Each  of  these  will  have  definable  attributes  and 
actions. 

The  use  of  OWL  (Web  Ontology  Language)  tool  should  be  considered  as  part  of  this 
effort  to  provide  information  interchange  into  and  out  of  the  ontology  as  requirements 
emerge. 


5.1.2  CO  NO  PS  Primitives  Language  Support 

Next,  a  small  set  of  representative  CONOPS  would  be  decomposed  to  determine  the 
additional  semantics  and  syntax  to  tie  the  primitives  into  a  language  that  will  form  the 
basis  for  simulation.  This  would  entail  an  investigation  of  potential  computing 
languages  to  represent  the  graphical  CONOPS,  while  understanding  the  capabilities  of 
potential  design,  debug  and  simulation  support.  In  addition,  this  would  include 
potential  language  and  tool  extensions. 


5.2  Mental  Model  Content 

Further  investigation  is  needed  to  ascertain  the  mental  model  content  most  appropriate 
for  each  phase  of  the  agile  CONOPS  process.  It  is  believed  that  multiple  mental  models 
will  exist  simultaneously  and  the  most  relevant  ones  will  be  task  dependent.  By 
conducting  cognitive  task  analysis  on  the  aforementioned  small  set  of  representative 
CONOPS,  we  can  (l)  understand  the  cognitive  demands  of  the  CONOPS  process,  (2) 
determine  similarities  and  differences  across  the  set  of  CONOPS  studied,  (3)  identify  a 
set  of  shared  mental  models  that  will  aid  the  team  in  expediting  the  agile  CONOPS 
process,  and  (4)  work  closely  with  the  graphical  CONOPS  developers  to  ensure  that  the 
interface  is  designed  in  a  manner  that  facilitates  the  development  of  the  identified 
shared  mental  models. 


5.3  ProofofConceftDemonstrator 

It  is  believed  that  it  should  be  possible  to  create  a  proof  of  concept  that  will  demonstrate 
a  graphical  CONOPS  capability  to  a  sufficient  level  using  an  existing  tool  such  as 
StarLogo  TNG  or  Microsoft  Surface  Table,  or  even  Siftable  Blocks.  This  would  be  a  proof 
of  concept  of  the  primitives/building  block  approach  to  visualizing  a  CONOPS.  An 
example  of  this  approach  is  to  decompose  a  portion  of  a  CONOPS  such  as  the 
Noncombatant  Evacuation  Operation:  Red  Cross  Rescue  Scenario  CONOPS  into  block 
primitives  as  described  above.  Once  defined,  a  proof  of  concept  simulation  could  be 
built  and  demonstrated. 
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It  would  be  necessary  to  determine  which  technology  would  be  used  to  accomplish  this 
task  early.  It  is  probably  less  important  which  technology  is  used,  and  more  important 
to  produce  demonstrator  to  further  the  learning  on  graphical  CONOPS  and  its  usage. 


5.4  Ability  to  Develop  Shared  Vision 

In  the  arena  of  shared  vision,  it  remains  to  be  proven  that  users  working  with  the 
approach  proposed  in  this  study  will  be  able  to  create  a  shared  vision  in  less  time  than 
using  conventional  methods.  Games  can  be  enlisted  as  metaphors  to  see  how  well  this 
might  be  accomplished.  Having  the  ability  to  see  outcomes  through  one’s  own  lens  and 
that  of  the  overall  group  might  make  it  easier  for  people  to  make  the  necessary  tradeoffs 
for  the  common  good.  The  ability  to  develop  shared  vision  should  be  evaluated  through 
the  use  of  early  prototyping  and  experiments. 

This  effort  can  be  accomplished  using  both  students  and  practitioners  to  acquire  the 
necessary  knowledge  and  data.  Additional  work  will  be  necessary  to  design  the  scenarios 
and  experiments  to  provide  useful  and  meaningful  data. 


5.5  On  the  FLy  Simulation 

One  challenge  is  the  ability  to  create  integrated  concept  simulation  models  based  on  the 
primitives,  semantics,  and  syntax  using  a  graphical  CONOPS  language.  Another  key 
challenge  would  be  to  change  the  model  on  the  fly  and  restart  simulations.  A  key 
constraint  on  the  modular  simulation  environments  is  that  all  the  potential  interfaces 
between  the  modules  are  pre-envisioned.  However,  if  new  connections  that  are  not 
previously  envisioned  are  created  the  outcome  of  the  simulation  may  not  be  predictable. 

For  instance,  ThinkCAD  (the  homegrown  LISP-based  tool  that  was  used  at  Thinking 
Machines)  was  restricted  in  the  sense  that  functionality  could  be  changed  on  the  fly 
during  simulation  (discussed  in  the  next  section),  but  structure  could  not.  However, 
there  was  much  that  could  be  done  even  within  these  constraints.  For  example,  one 
could  change  the  policies  by  which  decisions  are  made  and  go  back  in  history  and  rerun. 
So,  much  can  be  done  even  without  the  ability  to  change  structure  during  execution. 

The  primary  issues  these  environments  seem  to  run  into  are  I/O  compatibility  and 
maintaining  logical  consistency.  The  key  future  research  question  for  on  the  fly 
simulation  of  graphical  CONOPS  is  the  ability  to  allow  an  on-line  logic  checker,  in 
addition  to  an  interpreter  or  a  dynamic  compilation  capability,  to  permit  creation  of  new 
configurations. 

This  will  be  one  of  the  most  challenging  of  the  tasks,  and  is  dependent  on  the  previous 
research  tasks. 


Contract  Number:  H98230-08-D-0171  DO  001,10  002  RT3  CONOPS 

Report  No.  SERC- 2009- TR- 003 
October  30,  2009 


UNCLASSIFIED 

57 


UNCLASSIFIED 


6  Conclusions 


While  concepts  of  operations  (CONOPS)  are  still  generated  in  a  textual  format,  the 
growth  of  complexity  in  the  systems  and  system  of  systems  we  use  today  will  make  it 
ever  more  difficult  to  grasp  mentally.  As  a  simple  example,  a  modern  day  smartphone 
today  can  have  in  excess  of  8  million  lines  of  code  (LOC)  in  the  operating  system,  and 
over  22  million  LOC  of  application  code. 

Today’s  smartphone  will  serve  as  a  phone,  speakerphone,  a  communicator,  portable  file 
storage,  a  camera,  MP3  player,  personal  information  manager,  Internet  browser,  game 
platform,  and  GPS  device. 

That  phone  will  interface  with  cell  towers  via  a  cellular  radio,  Bluetooth  devices  via  a 
Bluetooth  radio,  other  USB  devices,  WiFi  connectivity,  connection  to  GPS  satellites,  and 
may  have  an  InfraRed  (IR)  interface.  The  software  will  interface  with  applications  on  a 
desktop  (Calendar  synchronization),  file  exchange,  music  player,  SMS,  FM  radio 
receiver,  and  Internet  access  to  cloud  computing.  It  will  have  a  camera  that  is  able  to 
take  pictures,  and  then  send  that  picture  via  email,  SMS,  or  to  a  connected  device. 

And  yes,  it  has  an  interface  to  the  user  -  a  human  user  -  and  any  two  individuals  may 
reason  about  the  same  task  in  totally  different  ways.  Part  of  the  interface  can  be 
implemented  using  hardware  in  the  form  of  mechanical  buttons,  and  some  interfaces 
will  be  implemented  in  software  through  virtual  buttons  and  displays. 

A  textual  CONOPS  does  not  do  a  smartphone  justice  in  describing  the  many  ways  the 
phone  will  be  used  by  the  individual.  This  problem  is  further  compounded  as  one 
considers  more  and  more  complex  systems,  with  more  and  more  users.  But,  if  one  does 
not  create  a  textual  CONOPS,  what  should  be  used  in  its  place? 

This  report  demonstrates  there  is  no  tool  currently  available  to  facilitate  an  alternative 
at  this  point.  There  are  technologies  that  can  be  applied  toward  the  desired  capability 
however.  This  report  should  form  the  basis  for  any  further  research  in  developing  a 
graphical  CONOPS  tool  to  assist  in  the  visualization  of  CONOPS. 

Finally,  the  authors  would  like  to  acknowledge  the  contributions  of  the  following 
individuals:  Gunnar  Feldman,  Research  Assistant;  Keith  Hall,  Research  Assistant;  and 
Mary  Bone,  Research  Assistant. 
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Appendices 


Appendix  A:  The  PLP  Heuristic 

Virtually  all  Complex  Large-Scale  Engineering  Systems  have  a  multitude  of  stakeholders 
and  would  benefit  from  some  level  of  stakeholder  participation  in  the  CONOPS  design 
process.  The  core  CONOPS  team  needs  to  identify  what  level  of  stakeholder 
participation  is  necessary  for  the  particular  system.  As  a  heuristic  tool,  we  have 
developed  the  PLP  (Participation  Level  Points)  heuristic  (shown  in  Table  n),  which 
links  system/stakeholder  characteristics  with  participation  levels.  The  premise  of  the 
PLP  heuristic  is  that  some  problem/ system  characteristics  increase  the  desired  level  of 
stakeholder  participation. 

The  PLP  heuristic  provides  a  direction,  not  an  answer.  As  such,  it  is  always  wiser  to  err 
on  the  side  of  higher  stakeholder  participation  than  to  settle  for  lower  stakeholder 
participation  levels.  If  the  PLP  of  a  system  is  4  or  higher,  stakeholder  participation  in 
the  conceptual  CONOPS  development  stage  is  necessary.  If  the  PLP  of  a  system  is  lower 
than  4,  then  it  probably  makes  more  sense  to  have  the  core  team  develop  the  CONOPS 
and  send  it  out  for  review  by  stakeholders.  Given  that  the  different  questions  in  Table  11 
do  not  necessarily  carry  the  same  weight  in  different  contexts,  decision-makers  need  to 
use  their  own  judgment  to  judge  whether  this  heuristic  is  appropriate  for  their  particular 
system. 
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lable  11:  The  PLP Heuristic 


Step  l:  Examine  System  Characteristics 

Yes 

No 

Is  the  system  in  question  spread  over  multiple  divisions/entities? 

l 

0 

Does  the  problem  affect  a  multitude  of  heterogeneous  stakeholder  groups  (developers, 
users,  managers,  others  outside  the  organization)? 

l 

o 

Has  the  informal  discussion  of  the  proposed  system  highlighted  the  potential  for 
disagreements  on  requirements/objectives? 

l 

o 

Are  cost  distribution  issues  among  different  organizational  important? 

l 

0 

Is  all  the  funding  necessary  for  building/managing  the  system  available  to  the  project 
developers? 

0 

1 

Is  there  significant  uncertainty  in  the  impact  of  design  decisions  on  system 
performance? 

l 

o 

Are  there  significant  intraorganizational  and  interorganizational  political  aspects  to 
the  system  in  question? 

l 

o 

Is  the  bringing  together  of  users,  developers  and  other  concerned  stakeholders  feasible 
logistically? 

l 

o 

Participation  Level  Points  (PLP) 

Sum 

Step  2:  Determine  Level  of  Participation  in  the  CONOPS  process 

PLP 

Inclusion  of  Stakeholders  in  all  Phases  of  CONOPS  development 

7-8 

Inclusion  of  Stakeholders  in  Conceptual  and  Requirements  Phases  of  CONOPS 
developments 

6 

Inclusion  of  Stakeholders  in  Conceptual  Phase  of  CONOPS  development 

4-5 

Development  of  CONOPS  by  core  team  with  stakeholder  commenting 

2-3 

Development  of  CONOPS  by  core  team 

0-1 
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Appendix  &  THE  SPK  Framework  for  Stakeholder  I demtihcation 


High 


Medium 


Low 


Stake  Power  Knowledge 


Decision-makers 
Highly  Organized  Stakeholders 
Experts  and  Expert  Agencies 
Less  Visible  Stakeholders 


Figure  21:  Tie  SPK  Fra  me  work 

Engineering  systems  development  often  impact  a  multitude  of  stakeholders,  some 
obvious,  some  less  so.  Given  the  limitations  on  how  many  stakeholders  can  physically 
participate  in  a  collaborative  CONOPS  process,  it  is  necessary  to  identity  the  critical 
stakeholders  for  such  a  process. 

Effective  stakeholder  identification  is  therefore  imperative  to  determine  who  will  be 
directly  or  indirectly  affected,  positively  or  negatively,  by  a  project  or  a  system 
management  plan,  and  who  can  contribute  to  or  hinder  its  success.  It  is  important  for 
the  project  sponsor/system  manager  to  be  comprehensive  in  identifying  and  prioritizing 
all  relevant  stakeholders,  including  those  that  are  not  usually  present  at  the  table 
(Susskind  and  Larmer,  1999).  Those  identified  will  then  need  to  be  consulted  to  varying 
degrees,  depending  on  their  impact  potential  on  the  system,  as  well  as  their  potential  to 
contribute  to  the  policy  process  through  knowledge,  resources  or  compliance  with 
implementation.  We  categorize  stakeholders  based  on  their  influence/power,  stake,  and 
knowledge. 

•  Decision-makers,  Managers  and  Principal  Users  (High  Stake,  High  Power, 
and  Differing  levels  of  knowledge):  Management  or  leadership  within 
organizations  and  high-leverage  systems  users  that  have  a  mandate  to  manage 
some  part  of  the  system  or  are  primary  users  of  the  system.  These  are  the  people 
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who  usually  create  a  demand  for  a  new  system  or  a  system  change  process  based 
on  their  perceived  needs. 

•  Stakeholders  with  financial/political  influence  (High  Stake,  Medium  to  High 
Power  and  Differing  Levels  of  Knowledge):  These  include  affected 
industry,  private  corporations,  landowners,  labor  unions,  nationally  recognized 
and  highly  organized  NGOs  and  other  groups  with  strong  political  influence. 

•  Knowledge-producers  (Low  Stake,  Low  Power,  High  Knowledge): 
Scientists,  Engineers  and  Consultants  working  in  the  academia,  technical 
consulting  firms,  local,  state  and  federal  science  agencies  and  scientific  and 
technical  offices  of  government  agencies  and  scientific  arms  of  NGOs  that  have  a 
stake  in  the  process,  but  have  no  specific  mandate. 

•  Other  affected  Stakeholders  (High  Stake,  Low  Power,  Differing  Levels  of 
Knowledge):  These  include  smaller  groups  of  stakeholders  directly  or  indirectly 
affected  by  system  management  strategies  or  the  proposed  project.  These  can 
include  less  organized  neighborhood  groups,  local  environmental  groups,  small 
business  owners  etc.,  depending  on  the  type  of  system  or  project  that  is  initiated. 

The  SPK  framework  provides  a  rough  mental  guideline  for  the  stakeholder  classification 
process.  Stakeholders  can  be  assessed  on  their  stake,  power  and  knowledge  (expert  or 
local)  on  the  decision.  Stakeholders  with  high  stakes  in  the  collaborative  process,  even  if 
they  lack  any  power  or  knowledge  can  add  legitimacy  and  community  acceptance. 
Stakeholders  with  high  knowledge  can  add  to  the  scientific/technical  /contextual 
validity  of  the  analysis,  while  stakeholders  with  power  (that  is  mandate  or  resources) 
can  increase  the  viability  of  the  process.  Stakeholder  with  lower  stake,  power  and 
knowledge  can  be  involved  through  feedback  systems,  information  websites,  media 
releases  and  outreach  campaigns. 

Of  course  it  is  important  to  realize  that  such  a  categorization,  while  useful  as  a  rough 
map,  should  not  be  the  exclusive  criteria  for  selecting  stakeholders  for  participation, 
given  that  even  smaller  actors  can  sometimes  be  effective  in  undermining  a  process. 
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Appendix  C :  shared  Menial  Models  (SMM) 

Mental  models  are  simplified  characterizations  humans  create  of  their  worlds  (Johnson- 
Laird  1983).  They  are  comprised  of  content  and  any  relationships  or  structure  among 
the  content  (Mohammed  et  al.  2000).  Individuals  use  them  to  describe,  explain,  and 
predict  their  surroundings  (Rouse  and  Morris  1986).  When  team  members  interact, 
their  mental  models  converge,  resulting  in  a  scenario  where  the  individual  team 
members’  mental  models  become  similar  to,  or  shared  with,  that  of  their  teammates’ 
mental  models  (McComb  2007;  McComb  and  Vozdolska  2007).  The  resulting  shared 
mental  models  can  be  defined  as  “knowledge  structures  held  by  members  of  a  team  that 
enable  them  to  form  accurate  explanations  and  expectations  for  the  task,  and  in  turn,  to 
coordinate  their  actions  and  adapt  their  behavior  to  demands  of  the  task  and  other  team 
members”  (Cannon-Bowers  et  al.  1993).  These  shared  mental  models  have  been 
routinely  linked  to  better  team  performance  (McComb,  2008).  As  such,  shared  mental 
models  are  a  critical  construct  for  consideration  when  examining  teams  facing 
operational  tasks  that  require  agility  such  as  mission  planning,  business  processes,  and 
intelligence  analysis. 

When  left  to  develop  naturally,  the  content  of  shared  mental  models  may  not  always 
represent  scenarios  that  will  positively  impact  team  performance.  For  instance,  evidence 
suggests  that  when  team  members  agree  that  they  are  cooperating,  their  performance 
deteriorates,  indicating  that  they  may  be  functioning  in  a  groupthink  mode  where  they 
do  not  question  each  others’  ideas  (McComb,  2007).  Under  these  circumstances,  teams 
may  benefit  from  a  strategically  designed  tool  to  guide  their  activities  and  their 
conversations.  Team  communication,  in  particular,  may  play  an  integral  role  in  the 
mental  model  convergence  process  (Kennedy  &  McComb,  in  press).  While  information 
processing  and  mental  model  updating  are  internal  processes,  the  communication  of 
information  occurs  external  to  the  team  member  and,  therefore,  represents  an 
observable  and  manipulatable  component  of  the  convergence  process. 

The  way  in  which  communication  needs  to  be  facilitated,  however,  is  dependent  upon 
the  (1)  type  of  task  being  undertaken  by  the  team  and  (2)  the  point  in  the  team’s  life 
cycle.  First,  communication  content  is  driven  by  the  type  of  task  being  undertaken.  For 
example,  Kennedy  and  McComb  (2007)  found  that  teams  working  on  an  intellective 
task  (i.e.,  a  task  with  a  correct  answer  that  is  technically  difficult  and  not  necessarily 
intuitive  (McGrath,  1984))  focused  their  conversations  on  teamwork  behaviors  needed 
to  organize  themselves.  As  such,  Kennedy  and  McComb  (2007)  investigated  mental 
model  convergence  about,  for  example,  how  the  team  would  approach  solving  the  task 
and  how  they  would  allocate  work  among  themselves  to  complete  the  task.  Alternatively, 
the  focus  of  the  team’s  task  may  be  on  generating  an  action-oriented  plan  (McGrath, 
1984).  McComb  et  al.,  (2009)  examined  teams  charged  with  creating  a  rescue  plan  for 
trapped  workers  on  a  hostile  island.  Team  members  were  assigned  specific  roles,  given 
contextual  information  about  the  mission,  enemy,  and  island,  and  capability 
information  about  personnel,  transport,  and  timing  options.  The  teams  had  to 
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assimilate  this  information  to  make  a  plan  that  drew  upon  specific  capabilities  under  the 
control  of  the  team.  Thus,  communication  exchanges  were  focused  on  sharing  the 
information  provided  and,  through  these  communication  exchanges,  the  team  members’ 
mental  models  about  the  task  converged. 

Second,  the  convergence  process  is  entwined  in  the  team  development  process  in  such  a 
way  that  the  topics  of  conversation  are  dependent  upon  the  purpose  of  the  team  at  a 
particular  point  in  time.  Indeed,  mental  model  convergence  may  occur,  to  various 
degrees,  across  all  stages  of  development  (McComb,  2007).  Upon  convergence  of  their 
mental  models,  team  members  will  use  them  to  guide  their  taskwork.  The  cognitive  shift 
to  taskwork  will  occur  automatically  as  the  team  members’  familiarity  with  the  mental 
model  content  increases  (Dutton,  1993;  Yoo  &  Kanawattanachai,  2001).  The  team  will 
work  on  their  taskwork  until  a  time  when  one  or  more  mental  models  need  revision  to 
accommodate  changes  in  the  circumstances  surrounding  the  team  (Smircich,  1983).  In 
other  words,  periodic  mental  model  maintenance  may  be  necessary.  When  maintenance 
is  needed,  each  individual  team  member  may  shift  to  an  individual  focal  level  where 
they  can  orient  themselves  to  the  new  information  available  and  re-differentiate  the 
information  they  have  (including  the  new  information  just  attained)  before  re¬ 
integrating  all  the  information  back  to  the  team  focal  level.  Kennedy  and  McComb 
(2009)  found  evidence  through  their  examination  of  communication  exchanges  of 
reconvergence  and  how  the  timing  of  reconvergence  impacts  team  outcomes.  Thus, 
monitoring  and  facilitating  team  communication  processes  can  assist  in  all  aspects  of 
the  mental  model  convergence  process. 
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Appendix  D:  Sample  Graphical  Representation  of  the  conceptual 
CONOPS  PHASE  FOR  AN  OFFSHORE  WIND  ENERGY  PROJ  ECT 

(Source:  Mostashari,  2005) 
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Major  Uncertainties 

(H=High  M=Medium 
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-  Visibility  of  Wind  Farm  (Strong) 

-  Actual  Turbine  Power  Generation  (Weak) 
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Energy 
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Appendix  E:  C  O  NO  PS  Elements/  C  o  mponentand  C  O  NO  PS  Proc  ess  Approac  h 


CONOPS  Project 

Type  of  CONOPS 

CONOPS  Process/Approach 

Process  Length 

Electronic  Records  Archives 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

1.  Initial  development  of  CONOPS  by  a  CONOPS 
development  team  utilizing  existing  documents  and  use- 
cases  as  a  way  to  extract  user  requirements. 

2.  Document  provided  to  relevant  stakeholders  for  comment 

3.  Final  CONOPS  document  issued  after  integration  of 
comments 

30  months 

Incident  Communications 
Interoperability 

Operational  Strategy 

N/A 

N/A 

Health  Monitoring  And 
Maintenance  Systems 

Products 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

Initial  Draft  by  CONOPS  Team  and  Revisions  based  on 
Comments  by  Stakeholders 

26  months 

Financial  System 
Modernization  Project 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

Office  of  the  Chief  Financial  Officer  (OCFO)  is  overseeing  the 
CONOPS  process,  and  revises  based  on  stakeholder  comments. 

9  months 

Systems  Engineering 

Education  Community  (Seec) 

Operational  Strategy 

EMWG  held  an  International  Workshop,  in  January,  1999  and 
produced  a 

Review  Draft  form  in  mid-1999  and  was  accepted  as  an  EMWG 
working  paper  in  the  January,  2000  workshop.  A  summary 
paper  was  included  in  the  INCOSE  2000  Conference 
Proceedings. 

12  Months 
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Missile  Early  Warning 

System 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

N/A 

N/A 

CONOPS  Project 

Type  of  CONOPS 

CONOPS  Process/Approach 

Process  Length 

Un  System  Response  In  An 
Influenza  Pandemic 

Operational  Strategy 

Not  available 

Not  available 

Grants.Gov  CONOPS 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

Not  Available 

Not  Available 

Civilian  Force  Development 

Operational  Strategy 

Mandate-based 

Not  Available 

Geostationary  Operational 
Environmental  Satellite 
(Goes) 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

N/A 

30  months 

Business  Enterprise 
Architecture  (Bea) 

Modification/  Upgrade/ 
Change  to  Existing 
System/Product 

N/A 

N/A 

Integrated  Ocean  Observing 
System  Data  Management 

And  Communications 

Operational  Strategy 

CONOPS  draft  used  as  a  stakeholder  review  process  document 

N/A 
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Appendix  F:  Annotated  Bibliography 

While  it  is  traditional  to  include  a  references  section,  we  have  chosen  to  provide  an 
annotated  bibliography  of  the  materials  that  were  investigated  for  this  study.  The 
advantage  of  an  annotated  bibliography  is  that  it  provides  a  short  annotation  about  the 
content  of  the  document  that  goes  beyond  the  title.  It  is  hoped  this  will  provide  the 
reader  with  better  information  in  determining  the  relevance  of  the  cited  document  to  the 
task  at  hand. 

American  Institute  of  Aeronautics  and  Astronautics.  ANSI/AIAA  Guide  for  the 

Preparation  of  Operational  Concept  Document,  G-043-1992.  ANSI/AIAA  G- 
043-1992. 1993. 

The  ANSI/AIAA  Standard  for  Operational  Concept  Documents. 

Ammala, Darwin.  A  New  Application  of  CONOPS  in  Security  Requirements 

Engineering.  CrossTalk:  The  Journal  of  Defense  Software  Engineering .  August  2000. 
Normally  used  for  describing  full  systems,  CONOPS  can  also  be  used  to  address  one 
single  aspect-such  as  security  in  a  large-scale  project.  This  paper  reports  on  a  recent 
Navy  contract  effort,  which  demonstrated  this  use  of  the  CONOPS.  This  paper  will 
describe  and  analyze  the  results  of  this  effort. 

Bizkevelci,Sezin  and  Cakmak,M.  A.  Technology  management  model  application  in 

concept  approval  decision  -  case  study:  Concept  of  operations  and  mission 
need  assessment  for  a  defence  system.  2008. 

This  paper  is  a  continuation  of  Research  &  Development  Project  Selection  Model  and 
Process  Approach  in  Defense  Industry  Related  Programs:  First  Phase  Concept  Approval 
Decision  study,  which  was  presented  in  PICMET07,  a  defence  industry  application  is 
realized.  The  application  contains  the  first  three  steps  of  Concept  Approval  Decision 
Phase.  First  of  all,  Concept  of  Operations  is  formed,  then  Mission  Need  Statements  are 
defined  and  finally  first  leg  of  Mission  Need  Analysis,  which  is  called  Functional  Area 
Analysis,  is  realized  for  a  generic  defence  system  as  an  example.  This  is  a  part  of  a  case 
study  and  it  is  aimed  to  demonstrate  the  applicability  of  the  theoretical  study  proposed 
in  PICMET07. 

Boardman,John;Sauser,B.;John,L.  and  Edson,R.  Conceptagon:  A  framework  for  systems 
thinking  and  systems  practice.  IEEE  International  Conference  on  Systems,  Man 
and  Cybernetics.  October  2009. 

This  paper,  building  on  the  body  of  Systems  Thinking  knowledge  and  cognizant  of  the 
key  problem-solving  systems  methodologies,  presents  a  new  framework,  the 
Conceptagon,  and  illustrates  its  use  by  describing  its  application  to  a  case  study  drawing 
on  the  United  States  Department  of  Defense  Biometrics  Enterprise. 

Booz  Allen  Hamilton.  Concept  of  Operations  (CONOPS)  Environmental  Protection 
Agency  Financial  System  Modernization  Project.  October  26,  2005. 

The  CONOPS  is  framed  by  EPA‘s  business  requirements  and  by  the  objectives  set  forth 
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by  the  Office  of  Management  and  Budget's  (OMB's)  Financial  Management  Line  of 
Business  (FM  LoB). 

Business  Transformation  Agency.  Concept  of  Operations  for  Business  Enterprise 
Architecture  (BEA)  Requirements.  September  14,  2007. 

This  CONOPS  identifies  and  describes  the  following  concepts,  relative  to  BEA 
development,  that  enable  the  BEA  to  address  the  two  types  of  requirements  or 
gaps/improvements:  a  “top  down  and  bottom  up”  ;approach  to  BEA  development  aimed 
at  delivering  the  right  balance  of  strategic  and  tactical  information  within  the  BEA, 
making  it  possible  to  address  the  strategic  and  tactical  requirements  and  federate  the 
BEA  with  relevant  component  and  system  architectures;  a  governance  model  and 
supporting  process  to  manage  the  priorities. 

Cannon-Bowers, J.  E.;Salas,E.  and  Converse, S.  Shared  Mental  Models  in  Expert  Team 

Decision-Making.  Individual  and  Group  Decision-Making:  Current  Issues.  Lawrence 
Erlbaum  Associates:  Hillsdale,  NJ.  1993. 

The  purpose  of  this  chapter  is  to  describe  how  the  notion  of  shared  mental  models  can 
advance  our  understanding  of  teamwork  and  team  decision  making,  and  to  delineate  the 
implications  of  adopting  such  a  position.  In  order  to  accomplish  this,  several  relevant 
bodies  of  research  reviewed,  including  literature  regarding  team  performance,  team 
decision  making,  and  mental  model  theory.  Following  this,  a  case  is  made  that,  when 
teamwork  is  conceptualized  in  terms  of  shared  mental  models,  it  provides  an  effective 
means  to  understand  this  rather  elusive  phenomenon.  Finally,  the  implications  of 
adopting  the  shared  mental  model  perspective  in  terms  of  team  decision  making  and 
training  for  team  decision  making  are  discussed 


Chen,Min;Lu,G.;Wen,Y.;Tao,H.  and  Su,H.  Study  on  the  Modelling  of  Geographic 

Conceptual  Scenario  Based  on  3D  Icons.  XXXVII.  Part  B4, Commission  TV.  July  3- 
11,  2008. 

There  are  many  problems  in  conceptual  modeling  process,  such  as  difficulties  in 
modeling  conception  sharing,  expressing  and  simulating  the  processes  via  a  visual 
method,  and  so  on.  To  solve  those  problems,  expression  meta  data  of  geographic 
conceptual  scenario  is  established  based  on  3D  Icons,  a  visual  conceptual  modeling 
approach  is  put  forward  to  realize  expression  and  construction  of  geographic  conceptual 
models  interactively.  By  the  experiments,  it  has  shown  that  our  research  can  accelerate 
practicality  of  geographic  conceptual  modeling  and  deepen  its  theoretic  meanings. 

Cloutier, Robert.  Model  Driven  Architecture  for  Systems  Engineering.  Conference  on 
Systems  Engineering  Research.  April  4-5,  2008. 

Model  Driven  Architecture  (MDA)  includes  a  model  based  approach  for  software 
architecture  and  design  as  well  as  a  set  of  key  principles  intended  to  improve  software 
interoperability,  reusability,  portability,  maintainability,  and  reliability.  This  paper 
reviews  the  key  MDA  principles  as  defined  by  the  Object  Management  Group,  and  then 
discusses  their  potential  applicability  to  the  systems  engineering  discipline,  along  with 
the  potential  benefits  to  systems  architecture  and  system  design  concepts  of  large  scale 
systems  development.  The  paper  will  identify  potential  MDA  practices  that  could 
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significantly  advance  the  practice  of  systems  engineering,  to  include  the  application  of 
system  design  patterns  to  system  architecture. 

Cloutier, Robert.  Model  Driven  Architecture  for  Systems  Engineers  -  Why  Should  We 
Care.  Telelogic  Users  Conference. 2006. 

Model  Driven  Architecture,  or  MDA,  is  an  Open  Management  Group  (OMG)  framework 
and  standard  with  the  goal  of  leading  “the  industry  towards  interoperable,  reusable, 
portable  software  components  and  data  models  based  on  standard  modelsi”.  When  one 
reads  the  MDA  literature,  it  is  full  of  IT  language  and  examples  relevant  to  software 
development.  So,  why  then  should  the  systems  engineering  community  be  interested  in 
this  approach?  Can  the  approaches  outlined  by  the  MDA  approach  be  used  to  perform 
systems  architecting/systems  engineering?  What  does  it  mean  to  create  a  compute 
independent  model  (CIM)  or  platform  independent  model  (PIM)  in  terms  of  systems 
engineering?  Finally,  is  this  approach  of  any  value  to  the  systems  engineering  discipline, 
and  how  can  TAU  help?  This  paper  will  attempt  to  address  those  questions. 

Cloutier, Robert  and  Verma,D.  Applying  the  concept  of  patterns  to  systems 
architecture.  Systems  Engineering.  10,2.  April  6,  2007. 

This  paper  provides  a  discussion  of  patterns  and  their  potential  applicability  to  complex 
system  architecting.  The  relevance  and  applicability  of  patterns  to  systems  architecting  is 
then  examined.  Research  with  regard  to  developing  a  pattern  form  for  documenting 
patterns  for  systems  architecting  is  presented,  and  this  is  demonstrated  on  a  command 
and  control  pattern,  using  both  IDEFo  and  UML.  The  application  of  this  pattern  within  a 
functional  architecture  is  then  explored.  Finally,  recommendations  for  the  development 
and  management  of  a  systems  architecting  and  architecting  pattern  repository  are 
offered. 

Cloutier, Robert  and  Verma,D.  Applying  Pattern  Concepts  to  Enterprise  Architecture. 

Journal  of  Enterprise  Architecture.  2,2.  May  2006. 

This  article  reviews  some  of  the  relevant  research  and  application  related  to  the  use  of 
patterns,  reviews  how  other  disciplines  are  using  patterns,  and  discusses  research  that 
has  been  done  on  applying  patterns  to  the  practice  of  architecting  complex  system 
(enterprise)  architectures.  Examples  of  architecture  patterns  are  presented  and 
discussed,  and  a  methodology  and  rationale  for  documenting  architecture  patterns  is 
presented. 

Cohen, Sholom.  Guidelines  for  Developing  a  Product  Line  Concept  of  Operations. 
Product  Line  Practice  Initiative.  August  1999. 

An  organization  develops  its  CONOPS  to  establish  the  desired  product  line  approach  that 
it  wishes  to  take.  These  guidelines  recommend  a  detailed  description  of  the  selected 
approach  and  possible  presentation  of  alternatives.  The  resulting  CONOPS  documents 
the  decisions  that  define  the  approach  and  the  organizational  structure  needed  to  put  the 
approach  into  operation. 

Commander  Air  Force  Space  Command.  Development  and  Use  of  Conceptual 
Documents.  10-606. 1996. 

This  Instruction  implements  AFPD10-6  Mission  Needs  and  Operational  Requirements, 
by  providing  guidance  and  procedures  for  developing  and  processing  Air  force  Space 
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Command  (AFSPC)  conceptual  documents.  It  provides  general  guidance  regarding  the 
development  and  use  of  the  various  types  of  conceptual  documents  in  support  of 
AFSPC's  mission  areas  and  systems. 

Darr, Stephen.  NASA  Aviation  Safety  &  Security  Program  (AvSSP)  Concept  of 

Operation  (CONOPS)  for  Health  Monitoring  and  Maintenance  Systems 
Products.  NIA  Report  #  2006-04.  September  2005. 

This  document  describes  current  and  proposed  practices  in  the  areas  of  fire  detection, 
fire  suppression,  health  status  monitoring  and  maintenance,  to  include  maintenance 
resource  management  in  the  broader  context  of  continuous  airworthiness  maintenance. 

Department  of  Defense.  Operation  Concept  Description.  DI-IPSC-81430. 1994. 

Department  of  Defense  Operation  Concept  Description  Standard. 

DiMario,Michael;Cloutier,R.  and  Verma,D.  Applying  Frameworks  to  Manage  SoS 
Architecture.  Engineering  Management  Journal.  20,4.  December  2008. 

This  article  addresses  some  of  the  reasons  why  the  management  approach  for  systems 
development  should  be  different  than  that  for  SoS  development  and  offers  a 
methodology  for  implementing  the  widely  accepted  Zachman  Framework  to  facilitate  the 
managing  of  SoS  architectures. 

Dutton,  J.E.  Interpretations  on  Automatic:  A  Different  View  of  Strategic  Issue 
Diagnosis.  Journal  of  Management  Studies.  1993. 

Models  of  strategic  decision-making  and  environmental  scanning  typically  assume  that 
decision-makers  diagnose  issues  actively,  using  conscious  and  intentional  effort  to 
identify  and  to  interpret  potentially  significant  events,  developments  and  trends.  This 
article  establishes  that  conditions  in  organizations  put  decision-makers  ‘on  automatic’  in 
their  diagnosis  of  strategic  issues,  with  direct  implications  for  the  process  and  content  of 
strategic  action.  Implications  for  theory  and  practice  are  established. 

Eden,C.  On  the  nature  of  cognitive  maps.  Journal  of  Management  Studies.  29,3.  May 
1992. 

A  discussion  into  the  development  and  use  of  cognitive  maps. 

El  Haouzi,Hind  and  Thomas, A.  A  Methodological  Approach  to  Build  Simulation 
Models  of  Manufacturing  Systems  with  Distributed  Control.  2005 
International  Conference  on  Industrial  Engineering  and  Systems  Management.  May 
16-19,  2005. 

Our  study  focuses  on  simulation  of  industrial  systems  with  distributed  control.  We 
propose  a  structured  approach  to  build  the  simulation  models.  This  approach  is  based  on 
methodology  ASDI  (analysis-specification-design-implementation)  and  it  is  independent 
of  any  platform  or  software  tool.  We  will  illustrate  the  use  of  this  approach  in  an 
assembly  line  manufacturing  application. 

Engineering  Talk.  Bentley  uses  Aesthetica  simulation  software.  Updated  March  5,  2009. 
Accessed  March  15,  2009.  http : / / www.engineeringtalk. com / news / ias /iasii.s.html. 
Bentley  Motors  has  implemented  Icona's  perceived  quality  simulation  software, 
Aesthetica,  at  its  design  and  manufacturing  plant  in  Crewe,  England. 
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Fairley, Richard  E.;Thayer,R.  H.  and  Bjorke,P.  The  Concept  of  Operations:  The  Bridge 
from  Operational  Requirements  to  Technical  Specifications.  The  First 
International  Conference  on  Requirements  Engineering .  April  18-22  1994. 

This  paper  describes  the  role  of  a  Concept  of  Operations  (CONOPS)  document  in  the 
specification  and  development  of  a  software-intensive  system.  It  also  describes  the 
process  of  developing  a  CONOPS,  its  uses  and  benefits,  who  should  develop  it,  and  when 
it  should  be  developed.  The  CONOPS  described  in  this  paper  is  compared  to  other  forms 
of  operational  concept  documents.  A  recommended  format  for  the  CONOPS  document  is 
presented  in  the  paper. 

Gabb, Andrew  P.  Operational  Concepts  -  Some  Variations.  Systems  Engineering,  Test  & 
Evaluation  Conference.  October  2002. 

This  paper  describes  different  types  of  operational  concept  documents  (OCDs)  that  may 
be  used  at  different  times  in  a  project.  While  several  guidelines  exist  for  the  development 
of  OCDs,  none  distinguish  between  the  different  stages  of  a  project  at  which  an  OCD  may 
be  developed,  nor  the  different  purposes  and  audiences  of  the  various  OCDs.  This  paper 
discusses  some  of  the  possible  OCD  variants  and  also  examines  the  differences  in  content 
of  the  different  variants. 

Gabb, Andrew  P.  Front-end  Operational  Concepts  -  Starting  from  the  Top.  Eleventh 

Annual  International  Symposium  of  the  International  Council  of  Systems  Engineering. 
July  1-5  2001. 

This  paper  addresses  the  need  for,  use  and  development  of  operational  concepts  at  the 
start  of  large  projects.  The  front-end  operational  concept,  derived  in  consultation  with 
the  system's  potential  users  and  other  stakeholders,  should  serve  as  the  driving 
document  for  the  acquisition  and  supply  of  the  system.  However,  there  is  often 
uncertainty  about  what  such  a  document  should  contain,  how  it  should  be  developed, 
and  its  relationship  to  other  project  documents.  The  paper  addresses  these  issues,  as  well 
as  the  evolution  of  the  Operational  Concept  throughout  the  life  of  the  project. 

Gilibert,Herve  and  Thevenot,R.  CONOPS  of  HALE  UTA  in  an  InfraRed  Early  Warning 
mission  for  Theater  Missiles  Defense.  AGARD  MSP  Symposium  on  “System 
Design  Considerations  for  Unmanned  Tactical  Aircraft  (LITA)”.  October  7-9, 1997. 

This  paper  presents  the  concept  of  High  Altitude  Lung  Endurance  UTA  equipped  with 
InfraRed  sensors  for  Tactical  Ballistic  Missiles  (TBM)  detection  and  tracking. 

Grants.gov.  Concept  of  Operations  Grants.gov  System.  April  17,  2007. 

The  Grants.gov  Concept  of  Operations  (CONOPS)  document  describes  the  desired 
characteristics  of  the  Grants.gov  system  from  the  users’  viewpoint.  This  document  also 
addresses  Grants.gov  requirements  for  operation  and  maintenance  (O&M)  of  the  system. 
This  CONOPS  briefly  discusses  the  Grants.gov  system,  its  components,  and  all  associated 
systems  that  impact  Grants.gov  operation. 

Greer, William.  Next  Generation  Modeling  of  Operational  Effects.  Science  and 
Technology  for  Chem-Bio  Information  Systems.  October  28,  2005. 

Extend  and  improve  the  CBRN  warfare  effects  on  military  operations  methodologies  and 
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transition  demonstrated  technologies  to  the  Joint  Operational  Effects  Federation  (JOEF) 
program. 

Haraldsdottir,Aslaug;Schwab,R.  W.  and  Alcabin,M.  S.  Air  Traffic  Management  Capacity- 
Driven  Operational  Concept  Through  2015.  2nd  USA/Europe  Air  traffic 
Management  R&D  Seminar.  December  1-4, 1998. 

This  paper  describes  an  approach  to  developing  an  operational  concept  for  the  US 
National  Airspace  System.  The  paper  illustrates  this  approach  by  defining  an  operational 
concept  that  is  driven  by  system  capacity  as  the  primary  performance  goal,  including  a 
logical  system  transition  path  from  the  current  system  to  a  higher  throughput  system  for 
2015.  In  addition,  this  paper  proposes  a  preliminary  design  process  for  the  NAS  that 
guides  the  system  architecture  design  through  a  careful  flowdown  of  performance 
requirements  and  operational  and  technology  trades. 

Herald, Tom  and  Verma,D.  Concept  of  Operations  Development  Considerations  for  a 
Highly  Networked  System.  Fourteenth  Annual  International  Symposium  of  the 
International  Council  on  Systems  Engineering.  June  20-24,  2004. 

Since  a  new  Net  Centric  System  (NCS)  is  often  made  up  of  existing  systems  plus  the 
development  of  new  state-of-  the-art  and  emerging-technology  systems,  there  is 
potential  to  provide  an  early  system  analysis  from  the  bottoms-up  and  integrate  with  the 
stakeholder  tops-down  needs.  This  source  of  information  is  the  existing  constituent 
system  Concept  of  Operations  (CONOPS)  documents.  However,  how  can  these  be 
effectively  used?  This  paper  focuses  on  a  reasonable  implementation  in  direct  support  of 
the  NCS  Systems  Engineer  with  the  tops-down  and  bottoms-up  mapping  for  effectively 
feeding  the  NCS-level  CONOPS  document  development  process. 

Hlupic,Vatka  and  Paul,R.  J.  Guidelines  for  selection  of  manufacturing  simulation 
software.  I IE  Transactions.  31,1.  January  1, 1999. 

This  paper  provides  guidelines  for  selecting  manufacturing  simulation  software 
according  to  the  intended  purpose  of  software  use.  The  basic  criteria  to  be  examined  in 
the  software  evaluation  process  are  listed  together  with  the  level  of  their  importance  for 
particular  types  of  users. 

IEEE.  IEEE  Guide  for  Information  Technology  -  System  Definition  -  Concept  of 
Operations  (CONOPS)  Document.  IEEE  Std  1362^-1998  (R2007).  1998. 

IEEE  CONOPS  Standard. 

Integrated  Computer  Engineering.  Concept  of  Operations  for  Electronic  Records 
Archives.  2004. 

This  Concept  of  Operations  (CONOPS)  document  provides  a  conceptual  overview  of  the 
proposed  ERA  system.  The  CONOPS  is  intended  to  support  the  evolution  of  a  fully 
integrated,  modernized,  and  functional  system  where  records  of  the  Federal  Government 
will  be  available  to  the  public  in  perpetuity.  Moreover,  the  CONOPS  is  a  living  document 
and  will  be  coordinated  in  a  collaborative  manner  with  industry,  public,  and  Government 
stakeholders  to  ensure  the  viability  of  the  concepts  represented. 

Jain,Rashmi.  Knowledge  Acquisition:  From  Concept  to  Competencies.  International 
Journal  of  Intelligent  Defence  Support  Systems. (unpublished) 
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Jain,Rashmi  and  Chandrasekaran,A.  A  Systems  Approach  to  Design:  Research  and 

Some  Results.  Asia-Pacific  Conference  on  Systems  Engineering.  September  22-23, 
2008. 

A  systems  approach  to  design  means  designing  from  a  holistic  perspective.  It  is  an 
approach  focused  on  understanding  the  functionality  for  which  the  system  is  designed 
for  by  keeping  the  focus  on  its  need,  context,  and  its  intended  lifecycle.  This  paper  is 
focused  on  the  need  and  current  challenges  of  teaching  engineering  students  a  systems 
approach  to  design.  The  paper  proposes  definitions  of  five  core  concepts  of  a  systems 
approach  to  design.  These  concepts  are  context,  abstraction,  trade-off,  inter¬ 
disciplinarity,  and  value.  The  paper  also  includes  discussion  on  the  findings  of  a  survey  of 
students  and  faculty  on  these  fundamental  concepts  of  system  design. 

Jain,Rashmi;Sheppard,K.;McGrath,E.  and  Gallois,B.  Promoting  Systems  Thinking  in 

Engineering  and  Pre-Engineering  Students.  American  Society  for  Engineering 
Education  Zone  1  Conference  Proceedings.  2008. 

This  paper  describes  vertically-integrated  curriculum  innovation,  in  which  graduate- 
level  coursework  spawned  a  pilot  program  to  embed  systems  in  a  core  engineering 
design  course  for  undergraduates  with  its  resulting  adoption  and  extension  to  a  core 
design  thread,  and  a  resulting  high  school  curriculum  development  and  dissemination 
effort  which  has  followed.  These  efforts  have  also  prompted  educational  research  to 
develop  the  academic  underpinnings  of  the  relatively  under-developed  scholarly 
foundations  of  systems  engineering. 

Johnson-Laird,  P.N.  Mental  Models:  Towards  a  Cognitive  Science  of  Language. 

Inference,  and  Consciousness.  Cambridge,  MA:  Harvard  University  Press.  1983. 
Mental  Models  offers  nothing  less  than  a  unified  theory  of  the  major  properties  of  mind, 
including  comprehension,  inference,  and  consciousness.  In  spirited  and  graceful  prose, 
Johnson-Laird  argues  that  we  apprehend  the  world  by  building  inner  mental  replicas  of 
the  relations  among  objects  and  events  that  concern  us.  The  mind  is  essentially  a  model¬ 
building  device  that  can  itself  be  modeled  on  a  computer.  This  book  provides  both  a 
blueprint  for  building  such  a  model  and  numerous  important  illustrations  of  how  to  do 
it. 

Johnston, Kevin  and  Ladd,J.  A  Concept  of  Operations  for  an  Integrated  Weather 

Forecast  Process  to  Support  the  National  Airspace  System.  11th  Conference  on 
Aviation,  Range,  and  Aerospace.  October  4,  2004. 

The  intent  of  this  CONOPS,  of  which  an  overview  is  provided  in  this  paper,  is  to  describe 
a  framework  for  an  improved,  standardized,  and  relevant  CWSU  operation  in  support  of 
the  NAS.  The  CONOPS  serves  as  a  guide  for  follow-on  efforts  to  establish  staffing 
requirements,  specific  procedures  and  directives,  as  well  as  determining  required 
changes  to  appropriate  planning,  requirements,  and  training  documents. 

Jost,Alan  C.  CONOPS:  The  Cryptex  to  Operational  System  Mission  Success. 

CrossTalk:  The  Journal  of  Defense  Software  Engineering .  October  2007. 

As  engineering  firms  start  to  design  any  number  of  systems  for  a  variety  of  customers 
and  end  users,  the  number  and  variety  of  system  documentation  can  be  overwhelming. 
Among  this  pile  of  documentation,  the  Concept  of  Operations  (CONOPS)  stands  out  as  a 
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critically  important  engineering  document  that  should  be  created  at  the  beginning  of  the 
system  development  and  maintained  throughout  the  engineering  life  cycle.  This  article 
discusses  the  CONOPS  and  if  it  truly  is  necessary  in  addition  to  all  of  the  other 
documentation  available. 

Karlsson, Goran  and  Wauthier,S.  Operational  Concept  Document:  A  process  based 

integration  of  production  IT  solutions  in  a  pharmaceutical  plant.  World  Batch 
Forum  North  American  Conference.  April  2001. 

The  operational  concept  document  is  a  result  of  the  analysis  of  the  manufacturing 
process  and  the  functions  completed  by  the  miscellaneous  IT  packages  that  bridge  the 
gap  between  the  pieces  of  equipment  and  the  business  systems.  This  presentation  will 
describe  the  method  used  to  perform  this  analysis  and  present  the  lessons  learned 
through  the  application  of  that  method  in  a  secondary  manufacturing  plant  greenfield 
project. 

Karplus,W.  J.  System  identification  and  simulation:  a  pattern  recognition  approach. 

Proceedings  of  the  Fall  Joint  Computer  Conference.  December  5-7, 1972. 

Recent  years  have  seen  continuing  and  increasingly-intensive  attempts  to  extend  the  art 
of  simulation  to  areas  which  heretofore  were  considered  too  complex  and  too  difficult  to 
lend  themselves  to  conventional  modeling  and  simulation  techniques.  The  extension  of 
simulation  techniques  developed  in  application  areas  such  as  control  system  design, 
electro-mechanical  systems,  etc.,  to  these  new  areas  has  often  been  disappointing,  if  not 
completely  unsuccessful.  This  is  due  to  the  difficulty  in  constructing  a  sufficiently-valid 
mathematical  model— a  model  which  can  be  used  for  prediction  with  a  reasonable 
amount  of  confidence.  It  is  well-known,  of  course,  that  even  under  the  best  conditions, 
inverse  problems  such  as  system  identification  problems,  do  not  have  unique  solutions. 
That  is,  inevitably  an  infinite  number  of  possible  models  will  satisfy  a  specified  set  of 
excitation/response  relationships.  Where  the  identification  process  is  further 
handicapped  by  uncertainties  as  to  system  structure  and  inadequate  experimental  data, 
the  pertinent  question  is  often  not:  “How  good  is  the  model?”  but  rather:  “Is  there  any 
point  to  modeling  at  all?”. 

Keel, Paul  E.  EWall:  A  visual  analytics  environment  for  collaborative  sense-making. 

Information  Visualization.  6, 1.  Spring  2007. 

We  introduce  EWall,  an  experimental  visual  analytics  environment  for  the  support  of 
remote-collaborative  sense-making  activities.  EWall  is  designed  to  foster  and  support 
'object  focused  thinking',  where  users  represent  and  understand  information  as  objects, 
construct  and  recognize  contextual  relationships  among  objects,  as  well  as  communicate 
through  objects.  EWall  also  offers  a  unified  infrastructure  for  the  implementation  and 
testing  of  computational  agents  that  consolidate  user  contributions  and  manage  the  flow 
of  information  among  users  through  the  creation  and  management  of  a  'virtual 
transactive  memory'.  EWall  is  designed  to  enable  individual  users  to  navigate  vast 
amounts  of  shared  information  effectively  and  help  remotely  dispersed  team  members 
combine  their  contributions,  work  independently  without  diverting  from  common 
objectives,  and  minimize  the  necessary  amount  of  verbal  communication. 
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Kennedy, Deanna  M.  and  McComb,S.A.  Team  Mental  Model  .Re-Convergence:  The  Effect 
of  New  Information  and  the  Mental  Model  Convergence  Process  during 
Collaborative  Activities.  INGroup  Conference.  2007. 

During  collaborative  activities,  new  information  may  arise  that  requires  adjustments  to, 
and  reconvergence  of,  mental  model  content.  Herein,  we  analyze  textual  data  collected 
during  team  experiments  to  explain  the  mental  model  convergence  process  when  new 
information  is  interjected.  Further,  we  examine  how  reconvergence  about  mental  model 
content  impacts  team  effectiveness. 

Kennedy, Deanna  M.  and  McComb,S.A.  Examining  the  Mental  Model  Convergence 

Process  through  Team  Communication.  Theoretical  Issues  of  Ergonomic  Science, 
(in  press). 

Mental  model  convergence  is  a  macrocognitive  process  critical  to  team  development.  Yet, 
little  is  known  about  how  the  convergence  process  occurs  in  a  team  domain.  To  date, 
researchers  have  examined  convergence  at  specific  points  in  time  after  performance 
episodes.  While  this  approach  has  moved  the  field  forward,  research  is  needed  that 
captures  the  dynamics  of  the  convergence  process  as  it  occurs  during  collaborative 
activities.  To  facilitate  this  needed  research,  we  present  a  framework  for  examining 
mental  model  convergence  via  communicated  mental  model  content.  Our  framework  is 
theoretically  based  on  relevant  research  in  cognitive  science  and  communication,  as  well 
as  more  recent  team  mental  model  research.  By  explicitly  establishing  the  theoretical 
connection  between  mental  model  convergence  and  team  communication,  we  extend  the 
extant  research  that  links  communication  and  team  cognition  and  address  a  gap  in  the 
team  mental  model  literature  regarding  how  to  examine  the  mental  model  convergence 
process  over  time. 

Lacroix, F.  W.;Button,R.  W.;Johnson,S.  E.;Chiesa,J.  and  Wise,J.  R.  A  Concept  of  Operations 
for  a  New  Deep-Diving  Submarine.  2002. 

The  NR-i  is  the  Navy’s  only  nuclear  deep-diving  research  submarine  capable  of  scientific 
and  military  missions.  Its  nuclear  reactor  will  be 

exhausted  in  2012;  therefore,  the  NR-i  must  be  refueled  or  retired  before  then.  As  part  of 
its  considerations  in  this  regard,  the  Navy  is 

developing  a  concept  of  operations  (CONOP)  for  a  possible  replacement  platform, 
initially  designated  the  NR-2. 

Lanner  Group  Limited.  Ford  Engine  Assembly  Line  Simulation  Tool.  2001. 

A  massive  amount  of  planning  and  investment  goes  into  every  new  engine  that  Ford 
Motor  Company  design.  In  today’s  highly  competitive  climate,  the  speed  to  market  for 
new  products  is  crucial  in  maintaining  competitive  advantage.  Ford  have  been  using 
WITNESS  since  the  mid-1980s  to  speed  up  the  design  of  their  engine  manufacturing 
facilities.  In  that  time  many  engine  assembly  lines  have  been  simulated,  with  the  model 
results  being  used  to  justify  design  decisions. 

Larson, Wiley;Sellers,J.;Kirkpatrick,D.;Thomas,D.  and  Verma,D.  Applied  Space  Systems 
Engineering.  McGraw  Hill,  2009. 

Law,Averill  M.  and  McComas,M.  G.  Simulation  of  manufacturing  systems.  Proceedings  of 
the  30th  conference  on  Simulation.  Winter  1997. 
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This  paper  discusses  how  simulation  is  used  to  design  new  manufacturing  systems  and 
to  improve  the  performance  of  existing  ones.  Topics  to  be  discussed  include: 
manufacturing  issues  addressed  by  simulation,  simulation  software  for  manufacturing 
applications,  techniques  for  building  valid  and  credible  models,  and  statistical 
considerations.  A  comprehensive  example  will  be  given  in  the  conference  presentation. 

Linebarger,John  M.;Spain,M.  J.  D.;McDonald,M.  J.;Spencer,F.  W.  and  Cloutier, R.  J.  The 
Design  for  Tractable  Analysis  (DTA)  Framework:  A  Methodology  for  the 
Analysis  and  Simulation  of  Complex  Systems.  International  Journal  of  Decision 
Support  System  Technology.  1,2.  April-June  2009. 

The  Design  for  Tractable  Analysis  (DTA)  framework  was  developed  to  address  the 
analysis  of  complex  systems  and  so-called  “wicked  problems.”  DTA  is  distinctive  because 
it  treats  analytic  processes  as  key  artifacts  that  can  be  created  and  improved  through 
formal  design  processes.  After  using  the  Systems  Modeling  Language  (SysML)  to  frame 
the  problem  in  the  context  of  stakeholder  needs,  DTA  harnesses  the  Design  Structure 
Matrix  (DSM)  to  structure  the  analysis  of  the  system  and  address  questions  about  the 
emergent  properties  of  the  system.  The  novel  use  of  DSM  to  “design  the  analysis”  makes 
DTA  particularly  suitable  for  addressing  the  interdependent  nature  of  complex  systems. 
The  use  of  DTA  is  demonstrated  by  a  case  study  of  sensor  grid  placement  decisions  to 
secure  assets  at  a  fixed  site. 

Lynott, Kevin.  NOAA’s  National  Weather  Service  Concept  of  Operations  River 

Forecast  Center  (RFC)  Analysis  and  Gridded  Forecast  Editor  Improvement. 

July  19,  2005. 

This  document  defines  the  River  Forecast  Center  (RFC)  analysis  and  gridded  forecast 
improvement  operational  concept.  This  document  includes  a  description  of  the  current 
system,  a  justification  for  the  proposed  change,  a  description  of  the  proposed  system, 
and  a  summary  of  anticipated  changes. 

Madni,Azad  M.;Lin,W.  and  Madni,C.  C.  IDEON:  An  extensible  ontology  for  designing, 
integrating,  and  managing  collaborative  distributed  enterprises.  Systems 
Engineering .  4,1.  2001. 

The  major  challenges  facing  organizations  are:  (a)  achieving  seamless  integration  of 
enterprise  design,  management  and  control  processes  and  supporting  applications;  (b) 
ensuring  interoperability  between  new  and  legacy  business  applications;  and  (c) 
adapting  business  strategies  and  ongoing  operations  to  changes  in  the  external  and 
internal  environments.  The  latter  requires  integrated  planning  and  execution  of 
enterprise  processes.  This  paper  presents  IDEONTM,  a  unified,  extensible  enterprise 
ontology  that  has  been  designed  in  response  to  these  needs.  Two  specific  applications  of 
IDEONTM  are  presented  along  with  the  specific  extensions  for  each  application. 

Madni,Azad  M.;Madni,C.  C.  and  Garcia, S.  K.  Cognitive  Model-enabled  Simulation-based 
Training  of  Aerospace  Operations  Center  Operators.  AIAA 
Inf otech@ Aerospace.  September  26-29,  2005. 

The  Aerospace  Operations  Center  (AOC)  is  a  weapon  system  for  the  command  and 
control  of  joint  forces  in  the  dynamic  battlefield  as  we  go  into  the  future.  The  training  of 
command  and  control  (C2)  personnel,  who  man  the  AOC,  continues  to  lag  because  of 
personnel  turnover  and  lack  of  cost-effective  “anytime,  anywhere”  training  tools.  The 
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challenge  in  today’s  compressed  personnel  assignment  cycle  is  to  develop  training  that 
satisfies  an  operator’s  individual  and  collaborative  decision  making  needs,  and  to 
provide  that  training  in  a  manner  that  rapidly  raises  the  learner’s  proficiency  to  a  level 
that  enables  the  learner  to  perform  effectively  in  the  operational  environment. 
Simulation-Based  Training  (SBT)  is  the  approach  of  choice  to  achieve  these  objectives. 
This  paper  presents  ProcessTrain™,  a  cognitive  model-  inspired  SBT  system  that  is  both 
cost-effective  from  a  development  perspective  and  scalable  in  terms  of  supported  users 
and  simulated  entities. 

Madni,Azad  M.;Samet,M.  G.  and  Freedy,A.  A  Trainable  On-Line  Model  of  the  Human 

Operator  in  Information  Acquisition  Tasks.  IEEE  Transactions  on  Systems,  Man 
and  Cybernetics,  12,4. 1982. 

A  trainable  model  of  the  human  operator  in  information  acquisition  tasks  is  described. 
The  purpose  of  this  model,  called  the  adaptive  information  selector  (AIS),  is  to  select  and 
present  textual  messages  automatically  to  users  in  computer-based  tactical  systems.  The 
AIS  was  implemented  and  tested  in  a  simulated  environment.  The  results  demonstrated 
the  model's  capability  to  1)  converge  on  distinctive  information  processing  strategies 
exhibited  by  different  operators;  and  2)  effectively  present  messages  in  their  order  of 
priority  for  each  operator.  The  AIS  is  potentially  useful  in  performing  information 
distribution  functions  in  command  and  control  systems  and  in  aiding  the  performance  of 
personalized  searches  of  large  data  bases. 

Maine  State  Office  of  Information  Technology  and  Maine  Emergency  Management  Agency. 

State  of  Maine  CONOPS  For  Incident  Communications  Interoperability. 

March  19,  2007. 

This  Concept  of  Operations  Plan  (CONOPS)  provides  guidance  to  public  safety  agencies 
(traditional  first  responders)  and  non-traditional  responders  for  developing  and 
employing  on-scene  interoperability  through  an  effective  Incident  Communications 
program. 

McComb,  Sara  A.  Mental  Model  Convergence:  The  Shift  from  Being  an  Individual  to 
Being  a  Team  Member.  Multi-Level  Issues  in  Organizations  and  Time.  2007. 

Mental  model  convergence  occurs  as  team  members  interact.  By  collecting  information 
and  observing  behaviors  through  their  interactions,  team  members’  individual  mental 
models  evolve  into  shared  mental  models.  This  process  requires  a  cognitive  shift  in  an 
individual's  focal  level.  This  chapter  presents  a  framework  describing  the  mental  model 
convergence  process  that  draws  on  the  extant  research  on  group  development  and 
information  processing.  It  also  examines  temporal  aspects  of  mental  model  convergence, 
the  role  of  mental  model  contents  on  the  convergence  process,  and  the  relationship 
between  converged  mental  models  and  team  functioning.  Preliminary  evidence 
supporting  the  framework  and  the  important  role  that  converged  mental  models  play  in 
high-performing  teams  is  provided.  The  chapter  concludes  with  a  discussion  of  the 
implications  of  this  mental  model  converence  framework  for  research  and  practice. 

McComb,  Sara  A.  Noncombatant  Evacuation  Operation:  Red  Cross  Rescue  Scenario. 

(unpublished) 

Hypothetical  Rescue  Scenario  documentation. 
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McComb,  S.A.  Shared  Mental  Models:  The  Convergence  of  Mental  Model  Content 
and  Structure.  Collaboration  and  Knowledge  Interoperability.  London:  Ashgate 
Publishing.  2008. 

The  purpose  of  this  chapter  is  to  examine  shared  mental  models  research  and  the  role  of 
shared  mental  models  in  the  macrocognitive  processes  of  teams.  To  this  end,  the  chapter 
begins  with  a  brief  overview  of  shared  mental  models  terminology  and  empirical 
research  results  demonstrating  their  essential  role  in  effective  team  functioning.  The 
next  section  describes  my  mental  model  convergence  conceptualization  and  provides 
preliminary  evidence  of  mental  model  convergence.  I  then  turn  my  attention  to  mental 
model  measurement  and  offer  some  suggestions  for  advancing  this  critical  aspect  of 
mental  model  research.  Finally,  I  examine  the  relationship  between  macrocognitive 
processes  and  shared  mental  models.  The  chapter  ends  with  implications  for  research 
and  practice  relating  to  the  study  of  shared  mental  models. 

McComb, Sara  A.;Green,S.  G.  and  Compton, W.  D.  Team  flexibility's  relationship  to 

staffing  and  performance  in  complex  projects:  An  empirical  analysis.  Journal 
of  Engineering  and  Technology  Management.  24,4.  2007. 

We  examine  the  role  of  flexibility  in  project  team  effectiveness.  Specifically,  we 
hypothesize  that  (1)  it  will  mediate  the  relationship  between  staffing  quality  and 
effectiveness  and  (2)  its  relationship  with  team  effectiveness  will  be  moderated  by 
project  complexity,  where  more  flexibility  will  be  required  when  projects  are  complex. 
Hypotheses  are  tested  using  data  collected  from  60  cross-functional  project  teams. 
Implications  for  research  and  practice  are  discussed. 

McComb,  Sara;  Kennedy,  D.;  Haas,  R.;  Warner,  N.  and  Letsky,  M.  Examining  Temporal 

Patterns  of  Mental  Model  Convergence:  Implications  for  Distributed  Teams 
Interacting  in  an  Electronic  Collaboration  Space.  ( working  paper)  2009. 

We  examine  the  effects  of  an  electronic  collaboration  space,  designed  specifically  to 
foster  information  processing,  on  teams’  cognitive  processes. 

McComb,  Sara  A.  and  Vozdolska,  R.P.  Capturing  the  Convergence  of  Multiple  Mental 
Models  and  Their  Impact  on  Team  Performance.  Southwestern  Academy  of 
Management  Conference.  March  13-17,  2007. 

We  examine  the  relationship  between  shared  mental  models  and  team  performance  by 
introducing  a  new  method  for  capturing  shared  mental  models  over  time  that  is  domain- 
independent  and  readily  applicable  to  the  field.  We  examined  seventy-two  teams  of  three 
undergraduate  students  during  two  sessions.  These  teams  completed  a  scheduling  task 
that  required  them  to  work  interdependently.  Our  results  indicate  that  (1)  the  method  we 
introduce  captures  mental  model  convergence  over  time,  (2)  multiple  shared  mental 
models,  which  are  not  domain  dependent,  exist  simultaneously  and  have  differing  effects 
on  team  performance,  and  (3)  the  content  of  the  mental  model  moderates  the  shared 
mental  model-team  performance  relationship.  Implications  for  research  and  practice  are 
discussed. 

McGrath,  J.  E.  Groups:  Interaction  and  Performance.  Englewood  Cliffs,  N.J.:  Prentice- 
Hall,  Inc.  1984. 

In  this  volume,  McGrath  examines  the  existing  group  literature,  describes  various 
methodologies  for  studying  groups,  and  presents  a  typology  of  tasks.  His  primary  focus  is 
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on  groups  as  vehicles  for  performing  tasks.  Therefore,  much  of  the  volume  is  devoted  to 
describing  eight  categories  of  tasks  and  the  research  that  has  been  conducted  on  groups 
engaged  in  those  tasks.  The  typology,  which  McGrath  labels  The  Group  Task  Circumplex, 
is  a  useful  tool  for  differentiating  among  types  of  activities  undertaken  by  groups  and 
teams. 

McIntyre, Cynthia.  U.S.  Manufacturing  -  Global  Leadership  Through  Modeling  and 
Simulation.  High  Performance  Computing  Initiative.  March  4,  2009. 

This  is  today's  headline:  The  Collapse  of  Manufacturing,  and  many  U.S.  manufacturers 
and  their  supply  chains  are  in  crisis.  In  this  time  of  crisis,  the  U.S.  has  the  technological 
tools  to  maintain  our  competitive  edge  and  global  leadership  in  manufacturing,  but  we 
risk  our  manufacturing  leadership  position  if  we  fail  to  utilize  the  game-changing  tool  of 
high  performance  computing  (HPC)  for  modeling,  simulation,  and  analysis. 

Miller, S.  and  Pegden,D.  Introduction  to  manufacturing  simulation.  2000  Winter 
Simulation  Conference  Proceedings.  2000. 

The  article  presents  an  overview  of  simulation  in  manufacturing  design  and  scheduling. 
A  review  of  the  modeling  considerations  in  both  application  areas  is  provided.  Finally,  a 
number  of  example  applications  are  presented  to  illustrate  the  concepts. 

Mixon/Hill, Inc.  StarTran  Automated  Vehicle  Location  System  Concept  of 

Operations.  Response  to  City  of  Lincoln  StarTran  RPF  05-053.  November 
2005. 

This  document  provides  a  Concept  of  Operations  (CONOPS)  for  the  City  of  Lincoln, 
Nebraska’s  StarTran  AVL  system.  The  CONOPS  document  is  designed  for  the  system 
owners,  system  users,  system  developers,  and  system  providers.  It  describes  the  current 
system  state,  establishes  the  need  for  system  change,  and  describes  the  proposed  system 
in  terms  of  features  and  functionality. 

Moertl,Peter;Beaton,E.  and  Viets,K.  En  Route  Merging  and  Spacing  Preparation 

concept  of  operations.  IEEE/AIAA  27th  Digital  Avionics  Systems  Conference,  2008. 
This  paper  describes  a  concept  of  operations  for  improving  the  merging  and  spacing 
operations  as  aircraft  approach  and  transit  into  the  terminal  area  that  implements  the 
NextGen  concept  for  en  route  and  arrival  operations. 

Mohammed,  S.;  Klimoski,  R.;  and  Rentsch,  J.R.  The  Measurement  of  Team  Mental 

Models:  We  Have  No  Shared  Schema.  Organizational  Research  Methods.  3.  2000. 
This  article  seeks  to  promote  the  advancement  of  empirical  research  on  team  mental 
models  by  (a)  highlighting  the  conceptual  work  that  must  precede  the  selection  of  any 
measurement  tool,  (b)  delineating  measurement  standards  for  group-level  cognitions, 
and  (c)  evaluating  a  set  of  techniques  for  measuring  team  mental  models.  Pathfinder, 
multidimensional  scaling,  interactively  elicited  cognitive  mapping,  and  text-based 
cognitive  mapping  are  critiqued  and  compared  according  to  their  treatment  of  content 
and  structure,  as  well  as  their  psychometric  properties.  We  conclude  that  these  four 
techniques  hold  promise  for  measuring  team  mental  models  and  illustrate  the  variability 
in  measurement  options.  However,  careful  attention  to  the  research  question  and 
research  context  must  precede  the  selection  of  any  measurement  tool. 
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NASA.  Project  Management:  Systems  Engineering  &  Project  Control  Processes  and 
Requirements.  4.1.3  Operational  Concept  Development.  March  2004. 

This  document  provides  a  description  of  the  basic  processes  and  general  practice  for  the 
development  and  operation  of  all  projects  managed  at  the  Lyndon  B.  Johnson  Space 
Center  (JSC). 

Nelson, Gary  G.  The  CONOPS  in  a  Self-Similar  Scale  Hierarchy  for  Systems 
Engineering.  Conference  on  Systems  Engineering  Research.  2007. 

The  standardized  CONOPS  differs  from  a  partial,  non-standardized,  but  widely  used 
"CONOPS".  This  paper  proposes  that  the  CONOPS  is  a  complete  kernel  for  any  system 
development  thread  and  scales  into  a  multi-level,  multi-peer  process  of  enterprise 
evolution.  Self-similar  versions  of  the  CONOPS  are  the  nodules  in  a  scale  hierarchy  that 
is  the  fundamental  architecture  of  complex  adaptive  enterprises.  This  architecture 
applies  to  risk  management  in  a  collective  of  agents  contending  over  limited  information 
and  other  resources.  This  "CONOPS-centric"  approach  should  displace  fragmented  and 
linear  approaches  to  complex  systems.  This  concept  has  firm  precedence  and  the  real 
puzzle  is  why  the  CONOPS  is  so  neglected  and  abused. 

Nichols, Ernie.  Developing  a  concept  of  operations:  enabling  improvement.  IEEE 
Proceedings  Aerospace  Conference.  7.  2001. 

INTELSAT,  an  international  telecommunications  satellite  business,  is  in  the  midst  of 
developing  and  maintaining  a  Concept  of  Operations  as  a  link  between  the  work  it  does 
and  the  vision,  mission  and  strategy  of  the  company.  This  paper  will  provide  a  short 
historical  summary  of  the  beginning  of  INTELSAT'S  Concept  of  Operations  and  how  it 
has  evolved.  It  will  show  how  work  processes  and  the  deliverables  they  produce  relate  to 
policy,  mission,  strategy,  and  objectives.  The  focus  of  the  discussion  will  be  how  the 
creation  and  maintenance  of  a  Concept  of  Operations  creates  value  throughout  the 
organization. 

NOAA.  Integrated  Ocean  Observing  System  Data  Management  and 
Communications  Concept  of  Operations.  October  2008. 

This  document  describes  the  initial  high-level  concept  of  operations  (CONOPS)  for  the 
DMAC  subsystem.  The  focus  of  the  document  is  to  define  the  functions  and  services  that 
IOOS  stakeholders  desire  the  DMAC  to  perform.  It  does  not  address  the  technology  or 
architecture  of  how  it  will  perform  those  functions  and  services. 

NOAA.  NOAA  Global  Earth  Observation  Integrated  Data  Environment  (GEO-IDE) 
Concept  of  Operations.  September  13,  2006. 

NOAA’s  GEO-IDE  is  envisioned  as  a  “system  of  systems”  -  a  framework  that  provides 
effective  and  efficient  integration  of  NOAA’s  many  quasi-independent  systems,  which 
individually  address  diverse  mandates  in  areas  of  resource  management,  weather 
forecasting,  safe  navigation,  disaster  response,  and  coastal  mapping  among  others. 

NOAA  and  NASA.  Geostationary  Operational  Environmental  Satellite  (GOES) 
Concept  of  Operations.  February  2008. 

A  CONOPS  for  the  upgrading  of  the  joint  NOAA/NASA  GEOS  System  to  improve  the 
nation’s  ability  to  monitor  and  forecast  weather  and  environmental  phenomena. 
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Pace, Peter.  Joint  Operations  Concepts  Development  Process.  CJCSI  3010.02B.  2006. 
This  instruction  provides  guidance  for  joint  concept  development  and  synchronizes  the 
efforts  of  the  joint  concept  community  in  the  DOD  capabilities-based  approach  to 
transformation.  Joint  concepts  link  strategic  guidance  to  the  development  and 
employment  of  future  joint  force  capabilities  and  serve  as  “engines  for  transformation” 
that  may  ultimately  lead  to  doctrine,  organization,  training,  materiel,  leadership  and 
education,  personnel  and  facilities  (DOTMLPF)  and  policy  changes.  This  instruction 
defines  the  specific  joint  concepts  known  as  the  Joint  Operations  Concepts  (JOpsC) 
family.  It  describes  how  these  concepts  are  developed  and  managed,  prescribes  specific 
concept  templates,  introduces  the  Joint  Concept  Steering  Group  (JCSG),  and  describes 
joint  experimentation  as  it  relates  to  assessment  of  the  JOpsC  family. 

Patrashkova,Ralitza  R.  and  McComb,S.  A.  Exploring  why  more  communication  is  not 
better:  insights  from  a  computational  model  of  cross-functional  teams. 

Journal  of  Engineering  and  Technology  Management.  21,1-2.  March  -  June  2004. 
Recent  evidence  suggests  that  communication  and  performance  in  cross-functional  new 
product  development  (NPD)  teams  are  curvilinearly  related,  but  fails  to  pinpoint  the 
reasons  for  this  relationship.  We  developed  a  computational  model  to  study  the 
communication  activities  of  cross-functional  new  product  development  teams.  Our 
simulation  confirms  the  recent  evidence  and  offers  insights  into  the  underlying  reasons 
for  the  curvilinearity.  We  provide  guidelines  regarding  when  the  top  performance  occurs, 
for  both  frequency  and  duration  of  synchronous  and  asynchronous  communication. 
Further,  we  perform  a  series  of  post-hoc  analyses  to  examine  the  reasons  for  the 
curvilinearity  of  the  communication-performance  relationship.  The  work  concludes  with 
a  discussion  of  the  theoretical  and  practical  applications  of  the  results. 

Patrashkova,Ralitza  R.;McComb,S.  A.;Green,S.  G.  and  Compton, W.  D.  Examining  a 

curvilinear  relationship  between  communication  frequency  and  team 
performance  in  cross-functional  project  teams.  IEEE  Transactions  on 
Engineering  Management.  50,3.  2003. 

In  this  work,  we  postulate  that  both  high  and  low  levels  of  team  communication  can 
impede  team  performance,  thus  leading  to  a  curvilinear  relationship  between  team 
performance  and  team  communication.  To  test  this  hypothesis,  the  relationships 
between  face-to-face,  e-mail,  and  telephone  communication  and  team  performance  were 
examined  for  60  cross-functional  project  teams. 

Peterman, Mike.  Simulation  Nation:  Process  simulation  is  key  in  a  lean 

manufacturing  company  hungering  for  big  results.  Quality  Digest.  May  2001. 

As  companies  continue  to  look  for  more  efficient  ways  to  run  their  business,  improve 
work  flow  and  increase  profits,  they  increasingly  turn  to  lean  manufacturing,  which  is 
used  by  best-in-class  operations  to  improve  their  processes,  achieve  their  goals  and  gain 
a  competitive  edge.  Process  simulation  has  become  an  increasingly  important  and 
integral  tool  as  businesses  look  for  ways  to  strip  nonvalue-adding  steps  from  their 
processes  and  maximize  human  and  equipment  effectiveness,  all  parts  of  lean 
manufacturing.  The  beauty  of  process  simulation  is  that,  while  it  complements  and  aids 
in  lean  manufacturing,  it  can  also  stand  alone  to  improve  business  processes. 
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Richards, Nicole  and  Edson,R.  System  Concept  of  Operations:  Standards,  Practices  and 

Reality,  nth  Annual  NDIA  Systems  Engineering  Conference.  2C7.  October  20-23, 
2008. 

Presentation  given  at  NDIA  SE  Conference  aimed  at:  Exploring  Industry  Use  of 
CONOPS,  Defining  a  quality  CONOPS,  Developing  Evaluation  Criteria  for  CONOPS 
goodness. 

Ring, Jack.  Concept  of  Operations:  Purpose  and  Practice,  (unpublished) 

This  material  is  intended  to  guide  authors  of  a  CONOPS  artifact  toward  successful 
fulfillment  of  their  commitments.  The  reader  will  find  guidance  regarding  the  What, 
How,  Who,  When,  Where  and  Why  of  CONOPS  preparation,  user  support,  and  value 
assessment  as  well  as  the  authors'  self-reflections  on  their  handiwork. 

Ring,  Jack  and  Wymore,A.  W.  Overview  of  a  CONOPS  for  an  SE  Education  Community. 

Tenth  Annual  International  Symposium  of  the  International  Council  on  Systems 
Engineering.  July  16-20,  2000. 

This  paper  summarizes  a  concept  of  operations  (CONOPS)  for  a  Systems  Engineering 
Education  Community  (SEEC).  An  SEEC  enables  those  involved  in  performing  systems 
engineering  to  improve  systems  engineering.  An  SEEC  is  envisioned  as  a  vehicle  for 
competency  and  proficiency  growth  by  all  practitioners  of  systems  engineering  as  well  as 
the  vehicle  for  measuring,  quantitatively,  both  the  supply  and  demand  sides  of  the  SE 
learning  community. 

Rouse,  W.B.  and  Morris,  N.M.  On  Looking  into  the  Black  Box:  Prospects  and  Limits  in 
the  Search  for  Mental  Models.  Psychological  Bulletin.  100. 1986. 

This  paper  explores  a  wide  range  of  issues  associated  with  research  on  mental  models. 
Based  on  a  functional  perspective,  mental  models  are  defined  as  the  mechanisms 
whereby  humans  generate  descriptions  of  system  purpose  and  form,  explanations  of 
system  functioning  and  observed  system  states,  and  predictions  of  future  system  states. 
Specifically,  this  paper  reviews  the  ways  in  which  different  domains  define  mental 
models,  characterize  the  purposes  of  such  models,  and  attempt  to  identity  the  forms, 
structures,  and  parameters  of  models. 

Sapre,Omkar.  Bentley  Systems  upbeat  on  Indian  Market.  Updated  November  17,  2008. 
Accessed  March  15,  2009. 

http://economictimes.indiatimes.com/articleshow/272076Q.cms?prtpage=i. 

When  Volkswagen  (VW)  starts  production  at  Pune  next  year,  it  will  be  one  of  its  fastest 
built  factory.  The  German  carmaker  built  its  facility  using  virtual  3D  tools  developed  by 
Bentley  Systems,  the  US-based  infrastructure  software  solutions  company. 

Scotti, Richard  S.  and  Gambhir,S.  A  Conceptual  Framework  for  a  Customer-centered 
System  Development  Lifecycle  Model.  Sixth  annual  International  Symposium  of 
the  International  Council  on  Systems  Engineering.  July  7-11 1996. 

This  paper  is  divided  into  two  parts.  The  first  contains  a  survey  of  various  system 
development  lifecycle  models,  including  the  Waterfall,  Spiral,  and  Evolutionary  Models. 
The  second  part  introduces  a  conceptual  framework  for  a  new  approach  to  complex, 
large-scale  system  developments  based  upon  a  customer  focus  throughout  the  system 
development  lifecycle.  This  customer-focused  approach  utilizes  automated  Concept  of 
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Operations  (CONOPS)  and  group  dynamics  techniques  to  allow  greater  understanding  of 
end-user  'actual'  needs  by  system  designers  via  more  productive  interactions.  This 
approach  is  based  on  the  development  and  use  of  detailed  user  objectives  and  critical 
success  factors  prior  to  and  separate  from  their  mapping  into  basic  and  derived  system 
requirements.  It  provides  a  means  for  capturing  previously  obscure  requirements  within 
the  user  environment  and  mindset. 

Siemens  ITS.  STARNET  Concept  of  Operations.  2006. 

This  Concept  of  Operations  document  is  intended  to  be  read  by  transportation 
operations  and  planning  personnel  at  STARNET  stakeholder  agencies,  and  by  any 
contractors  hired  to  assist  with  design,  implementation,  operation,  or  maintenance.  As 
with  nearly  all  documents  used  in  the  management  of  STARNET,  this  document  will  be 
used  throughout  the  life  of  the  system  and  will  be  updated  as  needed.  During  the 
planning  and  design  stage  of  STARNET,  it  reflects  the  stakeholders’  vision  for  how  the 
system  will  be  used  and  the  role  of  various  support  measures.  After  the  initial  system  is 
operational,  this  document  will  be  updated  to  reflect  how  the  system  is  actually  used,  to 
describe  the  support  measures  actually  in  place,  and  to  help  plan  expansion  or 
enhancements. 

Silva,L.;Ramos,A.  L.  and  Vilarinho,P.  M.  Using  simulation  for  manufacturing  process 
reengineering-a  practical  case  study.  Simulation  Conference  Proceedings,  2000. 
Winter.  2,2.  December  10-13,  2000. 

This  paper  presents  a  simulation  study  carried  out  to  solve  a  problem  of  manufacturing 
process  reengineering.  The  relevant  operational  performance  measures  were  analyzed  in 
order  to  allow  for  the  proposal  of  a  set  of  changes  to  the  actual  manufacturing 
operations. 

Smircich,  L.  Organizations  as  Shared  Meanings.  Organizational  Symbolism,  pp.  55-65. 
Greenwich,  CT:  JAI  Press.  1983 

The  overall  purpose  of  this  paper  is  to  illustrate  how  organizations  exist  as  systems  of 
shared  meanings  and  to  highlight  the  ways  in  which  shared  meanings  develop  and  are 
sustained  through  symbolic  processes.  The  paper  is  derived  from  an  ethnographic  study 
of  the  executive  staff  of  an  Insurance  Company.  It  describes  the  system  of  meaning  the 
group  members  used  to  make  sense  of  their  experience  and  traces  its  emergence  from 
their  interaction  and  its  influence  on  their  further  interaction.  The  paper  shows  how  such 
symbolic  processes  as  organizational  rituals,  organizational  slogans,  vocabulary,  and 
presidential  style  contribute  to,  and  are  part  of,  the  development  of  shared  meanings 
which  give  form  and  coherence  to  the  experience  of  organization  members. 

Smith, Brian.  Developing  and  Using  a  Concept  of  Operations  in  Transportation 
Management  Systems.  FHWA-HOP-oy-ooi.  2005. 

This  document  is  intended  to  serve  both  as  a  comprehensive  introduction  to  the  Concept 
of  Operations  and  a  reference  document  for  professionals  involved  in  developing  and 
using  a  Concept  of  Operations  for  Transportation  Management  Systems. 

Software  Engineering  Institute.  Concept  of  Operations  for  the  CMMI.  Carnegie  Mellon 
University.  January  15,  2001. 

This  Concept  of  Operations  (CONOPS)  for  the  CMMI  product  suite  includes  the 
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background  and  description  of  the  CMMI,  the  process  for  using  the  CMMI,  the  scenarios 
for  use,  the  process  for  maintenance  and  support  and  the  approach  for  adding  new 
disciplines.  It  is  intended  that  the  CONOPS  not  only  describe  the  use  of  the  proposed 
product  suite,  but  also  be  used  to  obtain  consensus  from  the  developers,  users  and 
discipline  owners  on  the  required  infrastructure  to  develop,  implement,  transition  and 
sustain  the  CMMI  product  suite. 

Spiegel, Rob.  Manufacturing  by  Computer  Simulation.  Automation  World.  October  2004. 
Recent  advances  in  factory  simulation  are  pushing  the  technology  beyond  its  core  use  for 
modeling  automation  to  also  provide  help  in  areas  ranging  from  training  and  product 
design  to  warehouse  management  and  supply  chain  planning. 

The  Boeing  Company  and  Lockheed  Martin  Corporation.  CONOPS  for  the  Systems  and 
Software  Test  Track. 

This  document  has  been  built  from  the  results  of  the  Systems  and  Software  Test  Track 
Phase  I  contracted  efforts  with  The  Boeing  Company  and  Lockheed  Martin.  This 
document  communicates  the  user  needs  and  the  expectations  of  the  proposed  system. 

Thompson, Ted.  Development  of  Operating  Concepts.  Fifth  annual  International 

Symposium  of  the  International  Council  on  Systems  Engineering .  July  22-26, 1995. 
Systems  engineering  techniques,  especially  requirements  collection  and  management, 
are  well  described  in  the  literature  but  appropriate  methods  of  determining  and 
documenting  operator  requirements  are  not  as  widely  discussed.  The  Canadian  Airspace 
System  (CAS)  is  a  large,  complex  systems  engineering  task.  This  paper  discusses  the 
methods  used  by  Transport  Canada  to  elicit,  review,  discuss  and  document  the  operating 
concepts  for  the  future  evolution  of  the  CAS. 

Thronesbery,Carroll;Molin,A.  and  Schreckenghost,D.  L.  Assisting  CONOPS  with 
Storyboards.  Houston  Human  Factors  Society  Meeting.  April  24,  2009. 

A  Concept  of  Operations  Storyboard  Tool  was  developed  to  assist  authors  in  building  a 
concept  of  operations  for  a  new  system,  refining  it  with  stakeholders,  and  using  it  to 
support  subsequent  development  activities.  We  illustrate  some  of  these  use  cases  to  show 
this  product-oriented  workflow  assistance  as  well  as  some  of  the  more  basic  storyboard 
support.  The  Storyboard  Tool  was  developed  iteratively,  testing  successive  prototypes  by 
using  the  tool  to  support  ongoing  research  and  development  projects  at  NASA  Johnson 
Space  Center.  In  addition  to  describing  and  illustrating  the  tool,  we  present  lessons 
learned  about  integrating  sketches  and  descriptions  for  clearer  communication,  the 
benefits  of  organizing  descriptive  information  as  structured  data,  and  assisting  the 
process  of  concept  development.  We  also  discuss  supporting  the  role  of  human  factors  in 
systems  engineering  and  the  value  of  iterative  development  for  systems  with  innovative 
human  task  support. 

Thronesbery,Carroll;Molin,A.  and  Schreckenghost,D.  L.  Concept  of  Operations  Storyboard 
Tool  Refinements  Based  on  Practical  Experiences.  IEEE  Aerospace  Conference. 
2008. 

A  storyboard  tool  has  been  prototyped  that  supports  authoring,  evaluating,  and 
exporting  a  concept  of  operations  and  its  illustrations.  A  requirements  engineering 
prototype  has  uncovered  new  requirements  to  communicate  the  intended  use  of  the 
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storyboard  tool.  New  designs  include  an  elaborated  help  system,  vertical  linking  of 
related  information,  and  a  work  plan  guide.  A  work  plan  guide  reveals  how  authors  can 
use  the  storyboard  tool  for  the  specific  needs  of  the  current  project.  We  discuss  the 
original  approach  and  design  along  with  the  newly  discovered  requirements  and  designs 
to  respond  to  them.  Finally,  we  present  lessons  learned  that  we  think  may  be  applied  to  a 
wide  range  of  software  development  projects. 

Thronesbery,Carroll;Molin,A.  and  Schreckenghost,D.  L.  A  Storyboard  Tool  to  Assist 

Concept  of  Operations  Development.  2007  IEEE  Aerospace  Conference.  March  3  - 
10  2007. 

We  describe  progress  in  developing  a  software  tool  to  help  create,  communicate,  and 
refine  concept  of  operations  (CONOPS)  information.  Information  found  in  a  CONOPS 
document  is  central  to  our  tool.  While  there  are  some  helpful  documents  prescribing 
CONOPS  format,  there  is  little  direct  assistance  to  express  the  concept  of  operations.  We 
describe  a  tool  to  help  author  this  information,  integrate  it  with  storyboard  sketches, 
refine  it  through  interaction  with  other  stakeholders,  and  finally  export  the  information 
in  a  form  directly  usable  by  developers.  We  have  developed  a  demonstration  prototype 
that  performs  the  core  functions  of  CONOPS  definition.  We  discuss  the  results  of  this 
prototyping  effort  and  plans  for  expanding  the  tool  functions. 

Tibbitts,Beth.  Flexible  simulation  of  a  complex  semiconductor  manufacturing  line 
using  a  rule-based  system.  IBM  Journal  of  Research  and  Development.  37,4. 1993. 
Rule-based  systems  have  been  used  to  produce  fast,  flexible  simulation  models  for 
semiconductor  manufacturing  lines.  This  paper  describes  such  a  rule-based  simulator  for 
a  semiconductor  manufacturing  line,  and  the  language  in  which  it  is  written.  The  model 
is  implemented  in  ECLPS  (Enhanced  Common  Lisp  Production  System),  also  known  as  a 
knowledge-based  or  expert  systems  language.  It  handles  very  large  models  (thousands  of 
data  elements,  or  more)  well  and  is  very  fast.  Subsequent  changes  improved  the  speed 
several  orders  of  magnitude  over  that  of  an  older  version  of  the  model,  primarily  through 
use  of  a  preprocessor  to  eliminate  duplicate  and  redundant  data,  and  by  enforcing  data 
typing  to  take  advantage  of  special  techniques  for  very  fast  processing  of  extremely  large 
matches  (hashed  indices). 

UN  System  Influenza  Coordinator  and  Pandemic  Influenza  Contingency  Team.  Concept  of 

Operations  for  the  UN  System  in  an  Influenza  Pandemic.  September  19,  2008. 
The  CONOPS  serves  as  an  overarching  framework  into  which  the  pandemic 
preparedness  plans  of  different  UN  agencies,  country  teams  and  missions  are  expected  to 
fit  in  the  event  of  a  Human  Influenza  Pandemic. 

Unknown.  What  is  a  CONOPS  anyway?  February  26,  2002. 

A  presentation  made  to  the  WMA  Chapter  of  INCOSE  on  February  26,  2002  about 
CONOPS. 

US  Air  Force.  Force  Development  CONOPS  Civilian.  January  2006. 

The  Civilian  Force  Development  Concept  of  Operations  (CONOPS)  is  part  of  a  series  of 
transformational,  strategic  level  Air  Force  (AF)  CONOPS.  It  outlines  a  cohesive  plan  for 
developing  the  civilian  component  of  the  Total  Force  The  strategy  is  to  blend  the  skills, 


Contract  Number:  H98230-08-D-0171  DO  001,10  002  RT3  CONOPS 

Report  No.  SERC- 2009- TR- 003 
October 30,  2009 


UNCLASSIFIED 

89 


UNCLASSIFIED 


knowledge,  and  experience  of  all  components  of  the  Total  Force  —  officer,  enlisted  and 
civilian  -  active,  Guard  or  Reserve  —  to  leverage  strengths  to  maximize  mission  success. 

US  Air  Force.  Air  Force  Smart  Operations  for  the  21st  Century  CONOPS.  Working 
Draft  Version  6.3. 

Air  Force  Smart  Operations  for  the  21  Century  (AFSO21)  is  our  Air  Force’s  dedicated 
effort  to  maximize  value  and  minimize  waste  in  all  of  our  environments  -  operational, 
support,  and  otherwise;  to  fully  integrate  continuous  improvement  into  all  we  do  across 
the  Air  Force.  AFSO21  is  our  standard  concept  and  approach  to  immediate  and  long¬ 
term  improvement.  This  CONOPS  articulates  what  is  required  throughout  the  Air  Force 
to  continue  to  assure  asymmetric  air,  space  and  cyberspace  capability  by  focusing  on  the 
processes  behind  our  core,  governing  and  enabling  processes  in  the  Air  Force. 

US  Department  of  Health  and  Human  Services:  Centers  for  Medicare  &  Medicaid.  Concept  of 
Operations  (CONOPS).  System  Lifecycle  Framework  Documentation.  January 
2005. 

US  Department  of  Health  and  Human  Services:  Centers  for  Medicare  &  Medicaid  Guide 
to  CONOPS  development. 

US  Department  of  Justice:  JMD:  Information  Resources  Management.  The  Department  of 
Justice  Systems  Development  Life  Cycle  Guidance  Document.  2003. 

Systems  Development  Life  Cycle  (SDLC)  emphasizes  decision  processes  that  influence 
system  cost  and  usefulness.  This  SDLC  establishes  a  logical  order  of  events  for 
conducting  system  development  that  is  controlled,  measured,  documented,  and 
ultimately  improved. 

US  Department  of  Transportation:  Federal  Highway  Administration.  Concept  of  Operations. 
SE  Guidebook  for  ITS. 

Objective,  Description,  Context  of  Process  and  FAQ's  about  US  DoT's  Federal  Highway 
Administration  CONOPS  Documents. 

US  Department  of  Transportation:  Federal  Highway  Administration.  Concept  of  Operations 
Template.  SE  Guidebook  for  ITS. 

Example  of  format  and  template  of  the  US  DoT  Federal  Highway  Administration's 
CONOPS  Documents. 

US  Marine  Corps.  Common  Logistics  Command  and  Control  System  Concept  of 
Operations.  April  19,  2004. 

The  purpose  of  this  document  is  to  provide  a  concept  of  operations  (CONOPS)  for  the 
Common  Logistics  Command  and  Control  System  (CLC2S).  The  document  will  describe 
conceptually  how  CLC2S  will  be  used  and  operated,  identify  standard  processes  and 
procedures  required  for  the  system  to  be  implemented,  and  delineate  additional 
opportunities  for  extended  use  and  functionality. 

Uygun,Ozer;Oztemel,E.  and  Kubat,C.  Scenario  based  distributed  manufacturing 

simulation  using  HLA  technologies.  Information  Sciences.  179,10.  April  29,  2009. 
This  paper  presents  an  overview  of  distributed  manufacturing  simulation  as  well  as  of 
information  representation  in  distributed  manufacturing  simulation  using  high  level 
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architecture  (HLA)  and  its  object  model  template  (OMT).  The  concept  is  explained  with 
a  scenario  which  is  provided  to  better  address  the  object  class  structure,  interaction  class 
structure,  attribute,  parameter  and  data  type  tables. 

van  Gaasbeek, James  R.  Operational  Concepts,  nth  Annual  NDIA  Systems  Engineering 
Conference.  3D2.  October  20-23,  2008. 

This  presentation  will  address  the  nature  of  the  operations  concept,  how  it  is  developed 
and  by  whom,  and  how  it  is  used  in  the  development,  deployment,  operations  and 
support  of  a  new  or  upgraded  system. 

Verma,Dinesh  and  Pennotti,M.  Fundamentals  of  Systems  Engineering.  Graduate 
Course  Notes.  Stevens  Institute  of  Technology.  2009. 

Volzdoska,Ralitza;McComb,S.  A.  and  Kennedy, D.  M.  Optimizing  Team  Performance 
through  Communication:  The  Role  of  Team  Size,  Interdependence  and 
Communication  Media.  Third  International  Conference  on  Maintenance  and 
Facility  Management. September  27-28,  2007. 

This  paper  extends  previous  research  that  established  a  curvilinear  relationship  between 
team  communication  and  performance  by  identifying  the  impact  of  team  size, 
interdependence  level,  and  media  choice  on  this  relationship.  Using  a  communication- 
performance  model  we  vary  team  size  and  interdependence  levels  across  asynchronous, 
mixed,  and  synchronous  media.  Implications  are  discussed. 

World  Intellectual  Property  Organization.  Concept  of  Operations  for  the  Reformed  IPC. 

IPC/CE/36/11,  Annex  X.  February  18,  2005. 

The  purpose  of  CONOPS  is  to  describe  the  classification  and  reclassification  process  of 
the  reformed  IPC  (International  Patent  Classification)  in  sufficient  detail  to  allow  all 
industrial  property  offices  to  understand  how  the  maintenance  of  the  classification  data 
of  the  core  and  advanced  levels  will  be  carried  out. 

Yoo,  Y.  and  Kanawattanachai,  P.  Developments  of  Transactive  Memory  Systems  and 
Collective  Mind  in  Virtual  Teams.  International  Journal  of  Organizational 
Analysis,  9,2.  2001. 

In  this  study,  we  examine  the  developments  of  transactive  memory  systems  and 
collective  mind  and  their  influence  on  performance  in  virtual  teams.  Building  on  an 
emerging  body  of  socio-cognitive  literature,  we  argue  that  transactive  memory  systems 
and  the  collective  mind  are  two  important  variables  that  explain  team  performance.  We 
tested  our  hypotheses  with  a  longitudinal  data  set  that  was  collected  from  38  virtual 
teams  of  graduate  management  students  from  six  universities  in  four  countries  over 
eight  weeks.  The  results  suggest  that  the  influence  of  team  members'  early 
communication  volume  on  team  performance  decreases  as  teams  develop  transactive 
memory  systems  and  a  collective  mind.  The  results  further  suggest  that  the  development 
of  a  collective  mind  represents  a  high-order  learning  in  team  settings. 

Young, Eric  C.  The  U.S.  Navy’s  Role  in  Executing  the  Maritime  CONOPS  for  U.S. 
Homeland  Security/Defense.  Naval  War  College.  Masters  Thesis.  2002. 

The  proposed  U.S.  HLS/D  maritime  CONOPS  comprises  Prevention,  Defense,  and 
Response  phases  and  discusses  how  best  the  Navy  may  perform  the  relevant  maritime 
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tasks  in  each  phase.  Maritime  tasks  include:  deterring  enemy  threats  against  the 
homeland;  defending  against  enemy  attacks;  supporting  civil  authorities;  security 
cooperation  and  contingency  response  with  Canada  and  Mexico;  and  command  and 
control  coordination. 
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